pixidock_template

Philosophy - Hermetic, Reproducible, and Neat Python environment(s)

This template shamelessly targets only 64 bit linux systems while favoring NVIDIA GPUs and Ubuntu. Sorry! PRs are welcome for support for other platforms ;p

Also, this template includes environments containing popular libraries for roboticists, as I am a roboticist, and I created this template largely for myself. However, it really could be used for any Python project, especially if interfacing with the GPU. The robotics environments can easily be deleted; get rid of the envs/deps you don't need! For those who like robots, there are environments for PyTorch, PyTorch3d, ROS 2, ROS 2 with GPU, Genesis Simulator, NVIDIA Isaac Lab (NVIDIA Isaac Sim with PhysX physics), and Isaac Lab Newton (Isaac Lab with NVIDIA's Warp-based Newton physics engine for GPU-accelerated simulation). Check out the pixi.toml to see all environments.

Pixi and Docker are two tools that together, in my opinion, can create a largely hermetic, reproducible, and neat Python environment. I think it's better than just pip and/or apt installing, or using virtualenv, or uv alone, conda, or mamba, or other virtualization tools. I will give Podman and Bazel an honorable mention. Of course, not all included environments are perfectly hermetic, reproducible, and neat, as some things need to be configured after Pixi environment creation, resulting in some instability as dependencies are sometimes omitted from the Lockfile. That being said, it is the closest I've gotten to hermetic, reproducible, and neat for many of the environments.

Although Pixi can create hermetic environments largely on its own, oftentimes a thin Docker virtualization layer may be needed to get CUDA to be reproducible across mismatching host CUDA versions to be able to use the GPU (see an example here)

I also like to use Pixi with ROS 2 through the RoboStack project, as this allows for using specific versions of PyTorch+CUDA with ROS 2 that may not be packaged with the apt repository ROS 2 package versions.

For example, a certain ROS 2 apt library named library A may be compiled against a specific libtorch version A when packaged in apt, while an interesting third-party machine library named library B may depend on libtorch version B. In this case, what some users may do, is try to just to globally pip install torch==version B. However, this can lead to an undefined symbol problem when trying to use both library A and library B together as library A was compiled against libtorch version A. Using RoboStack with ROS, allows to try to find a version of library A that depends on library B to avoid this sort of incompatibility issue. It also increases reproducibility as well with versioned lockfiles for ROS packages.

In some cases, certain packages/libraries may not be compatible out of the box with RoboStack. In this case, there are a few good options on make the package/library compatible.

Option A: Help everybody by adding the package/library to RoboStack

Option B: Help everybody by opening an issue with a package request on RoboStack on your ROS distribution

Option C: Building the library within a ROS workspace with RoboStack

The desired library may not be available on the RoboStack package index, but it maybe can still be built as part of the ROS workspace. Run pixi r build-ros to build the synchros2 package from source in ros2_ws directory to see an example. However, be wary of relying on rosdep.

Option D: Running the library within Docker with it's own standalone version of ROS 2, that communicates through ROS 2 with this template package

In a time crunch, it may be useful to use an external library as-is without modifications. In this case, I would advise running these libraries in their own standalone docker container, using the network=host flag when starting the container, with ROS 2 installed from apt or built from source. This way, the library within the container should hopefully still be able to communicate with this template's ROS despite originating messages from two different versions of ROS 2. Be wary of ufw blocking the UDP packets; see how to enable multicast.

Finally, Pixi has great multi-environment support. For example, the same computer vision for robotics package code can be used without ROS, with ROS, with a simulator, with a GPU, or combinations of the previous mentioned environments, all while sharing the same PyTorch version as a dependency.

Quick Start

Usage as Template

git clone https://github.com/garylvov/pixidock_template <YOUR_PROJECT_NAME>
cd <YOUR_PROJECT_NAME>
bash scripts/name.bash <YOUR_PROJECT_NAME>

Then, delete everything above Quick Start, this line, and edit the rest of the README. Feel free to open issues on the template GitHub page. Set a license if applicable and delete that section from this README. Good luck!

Clone the Repo and Install Dependencies

This repository primarily targets x86-64 Linux.

git clone <YOUR_REPO_LINK>
cd TEMPLATE && git submodule update --init --recursive

Pixi (version 0.62.2) is all that's needed to locally to resolve dependencies if:

• Using CPU only environments on x86-64 Linux
• Using GPU compatible environments while having a matching CUDA version installed on the x86-64 Linux host system (CUDA ver. 12.6)

Otherwise, Docker (and Docker Compose) can be used to address GPU driver version mismatches through virtualization, while autoinstalling all dependencies. This approach ensures the environment matches the project requirements exactly if possible. To configure an NVIDIA GPU system with Docker/Docker Compose, refer to this guide.

Installing dependencies through Pixi locally WITHOUT Docker is highly recommended.

To use Pixi locally on the base system run the following installation command.

curl -fsSL https://pixi.sh/install.sh | bash

If using Docker, install Docker and Docker Compose on the host system, and enter the container prior to running any commands with the following.

# Build and enter the container
bash scripts/develop-compose.bash
# For a new terminal, run "docker exec -it TEMPLATE_dev bash"
# To stop, run "docker compose down && sudo xhost -"

Environment Entry and Configuration

Then, from within either the project parent folder or the Docker image home directory, run the following to activate the environment(s).

# You may need to increase ulimits before running GPU-intensive tasks:
# ulimit -n 64000  # Increase file descriptor limit
# ulimit -u 8192   # Increase process limit (helps for multi-GPU training)

pixi s  # Activate environment, add -e for specific env,
# Envs: gpu|ros2-gpu|ros2-cpu|genesis-gpu|genesis-ros2-gpu|isaaclab-gpu|isaaclab-ros2-gpu|isaaclab-newton-gpu


# On some systems (e.g., RHEL 9.4), you may need the following
# export CONDA_OVERRIDE_GLIBC=2.35
# For Isaac Lab (PhysX)
pixi r -e isaaclab-gpu install-isaaclab
# To use: pixi s -e isaaclab-gpu
# For Isaac Lab Newton (Warp-based)
pixi r install-isaaclab-newton
# To use: pixi s -e isaaclab-newton-gpu

# For ROS, build the ros2_ws (colcon build is auto-configured by ros2_ws/colcon-defaults.yaml)
pixi r build-ros
# To build your package, clone it into ros2_ws/src, and add its name to the colcon-defaults.yaml
# To use: pixi s -e ros2-gpu or ros2-cpu
# Genesis + Isaac Lab environments are still a bit flaky despite my best efforts ;(

Run Commands

All run commands must occur from within the project parent folder or Docker image home directory to function correctly.

Here is where to put the entrypoints your user may care about.

Community Contributions

If something doesn't work, please create a GitHub issue! PRs are always welcome!

Testing and Linting

pixi run test  # Run with test environment
pixi run lint

Changing License

1. Update LICENSE.txt file with your new license
2. Run: pixi run lint.

Acknowledgement

This repository was created from the pixidock_template created by Gary Lvov, under fair use of the BSD 1-Clause License