How to Install Spinning Up and Run Experiments

Updated

Spinning Up is OpenAI's educational deep RL package, and getting from zero to a running experiment takes three installs (Python, OpenMPI, Spinning Up itself) plus one command to launch training. The documentation's Installation page covers Python, OpenMPI, and Spinning Up, with MuJoCo listed as optional; the Running Experiments page covers launching from the command line or from scripts, where outputs are saved, and how to plot results. Follow the order below, since Spinning Up's MPI-based launcher depends on OpenMPI being present first.

Before you start

  • You need a working Python environment. The docs treat "Installing Python" as its own step, so do that before anything else.
  • OpenMPI is a separate, required install — Spinning Up's experiment launcher uses MPI tools, so skipping this will break spinup.run.
  • MuJoCo is explicitly marked optional. You only need it for the environments that depend on it; the core algorithms and the install verification do not require it.
  • The docs include a "Check Your Install" step. Run it before launching a real experiment so you find problems while the fix is still obvious.

Install in this order

1. Python

Install Python following the documentation's Installing Python page. This is the base requirement for everything else.

2. OpenMPI

Install OpenMPI per the Installing OpenMPI page. Do this before Spinning Up, because the package's MPI Tools and experiment launcher rely on it.

3. Spinning Up

Install the package itself following the Installing Spinning Up page. Once installed, run the "Check Your Install" step from the docs to confirm the pieces are wired together correctly.

4. MuJoCo (optional)

Only if you need it: follow Installing MuJoCo (Optional). If you skip this, expect MuJoCo-dependent environments to be unavailable — that is the intended trade-off, not a broken install.

Run an experiment

The docs give two ways to launch training, and both are covered under Running Experiments:

  • Launching from the Command Line — the direct route for a single run.
  • Launching from Scripts — the route to use when you want to configure or repeat runs programmatically.

Either way, the input is an algorithm plus its configuration, the action is the launch command, and the expected result is a training run whose progress and outputs land in the save directory described below.

Where outputs go, and how to plot them

  • Algorithm Outputs and Save Directory Location are documented under Experiment Outputs. That is where to look to find what a finished or in-progress run wrote to disk.
  • Loading and Running Trained Policies is also under Experiment Outputs — this is how you take a saved policy and actually run it.
  • Plotting Results is its own page. Use it to turn the logged outputs into readable curves rather than inspecting raw files.

The Logger and Plotter utilities documented under Utilities Docs are the underlying tools behind these steps.

Common snags

  • Launch fails immediately with MPI errors — OpenMPI was skipped or installed after Spinning Up. Revisit the Installing OpenMPI step.
  • An environment won't start — check whether it depends on MuJoCo, which is optional and may not be installed.
  • You can't find your results — go back to Save Directory Location rather than searching the filesystem blindly.
  • Training ran but you can't evaluate it — use Loading and Running Trained Policies; a saved run is not automatically a runnable policy.

What to read next

Once a run completes end to end, the docs' Introduction to RL (Parts 1–3) and the Algorithms Docs for VPG, TRPO, PPO, DDPG, TD3, and SAC explain what you just ran. The Benchmarks for Spinning Up Implementations page gives reference performance per environment so you can judge whether your run behaved as expected.