> For the complete documentation index, see [llms.txt](https://slm-lab.gitbook.io/slm-lab/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://slm-lab.gitbook.io/slm-lab/benchmark-results/continuous-benchmark.md).

# Continuous Benchmark

## MuJoCo Benchmark Results

SLM Lab v5.2 validates PPO, SAC, and CrossQ on [Gymnasium MuJoCo environments](https://gymnasium.farama.org/environments/mujoco/). MuJoCo (Multi-Joint dynamics with Contact) provides physics simulation for continuous control tasks ranging from simple pendulums to complex humanoid locomotion.

Results below are from January–March 2026 benchmark runs using MuJoCo v5 environments.

All trained models and metrics are publicly available on [HuggingFace](https://huggingface.co/datasets/SLM-Lab/benchmark).

### Methodology

Results show **Trial-level** performance:

1. **Trial** = 4 Sessions with different random seeds
2. **Session** = One complete training run
3. **Score** = Final 100-checkpoint moving average (`total_reward_ma`)

The trial score is the mean across 4 sessions, providing statistically meaningful results.

### Standardized Settings

| Category | num\_envs | max\_frame | log\_frequency | ASHA grace\_period |
| -------- | --------- | ---------- | -------------- | ------------------ |
| MuJoCo   | 16        | 4e6-10e6   | 10000          | 1e5-1e6            |

The `grace_period` is the minimum frames before ASHA early stopping can terminate underperforming trials.

**Algorithms**: PPO, SAC, and CrossQ. Network: MLP \[256,256], orthogonal init. PPO uses tanh activation; SAC and CrossQ use relu. CrossQ uses Batch Renormalization in critics (no target networks).

**Note on frame budgets**: SAC uses higher update-to-data ratios, making it more sample-efficient but slower per frame than PPO (1-4M frames vs PPO's 4-10M). CrossQ uses UTD=1 (like PPO) but eliminates target network overhead, achieving \~700 fps — its frame budgets (3-7.5M) reflect this speed advantage. Scores may still be improving at cutoff.

{% hint style="warning" %}
**v5 vs v4 Difficulty:** Gymnasium MuJoCo v5 environments are significantly harder than v4. Key changes include:

* Updated physics engine with more accurate contact dynamics
* Revised reward functions with stricter success criteria
* Termination conditions more closely match real-world failure modes

Expect **10-30% lower scores** compared to v4 benchmarks. See [Gymnasium Migration Guide](https://gymnasium.farama.org/content/migration-guide/) for details.
{% endhint %}

### Spec Files

**Spec Files** (one file per algorithm, all envs via YAML anchors):

* **PPO**: [ppo\_mujoco\_arc.yaml](https://github.com/kengz/SLM-Lab/blob/master/slm_lab/spec/benchmark_arc/ppo/ppo_mujoco_arc.yaml)
* **SAC**: [sac\_mujoco\_arc.yaml](https://github.com/kengz/SLM-Lab/blob/master/slm_lab/spec/benchmark_arc/sac/sac_mujoco_arc.yaml)
* **CrossQ**: [crossq\_mujoco.yaml](https://github.com/kengz/SLM-Lab/blob/master/slm_lab/spec/benchmark/crossq/crossq_mujoco.yaml)

**Spec Variants**: Each file has a base config (shared via YAML anchors) with per-env overrides:

| SPEC\_NAME                    | Envs                                           | Key Config                                         |
| ----------------------------- | ---------------------------------------------- | -------------------------------------------------- |
| ppo\_mujoco\_arc              | HalfCheetah, Walker, Humanoid, HumanoidStandup | Base: gamma=0.99, lam=0.95, lr=3e-4                |
| ppo\_mujoco\_longhorizon\_arc | Reacher, Pusher                                | gamma=0.997, lam=0.97, lr=2e-4, entropy=0.001      |
| ppo\_{env}\_arc               | Ant, Hopper, Swimmer, IP, IDP                  | Per-env tuned (gamma, lam, lr)                     |
| sac\_mujoco\_arc              | (generic, use with -s flags)                   | Base: gamma=0.99, iter=4, lr=3e-4, \[256,256]      |
| sac\_{env}\_arc               | All 11 envs                                    | Per-env tuned (iter, gamma, lr, net size)          |
| crossq\_mujoco                | (generic base)                                 | Base: gamma=0.99, iter=1, lr=1e-3, policy\_delay=3 |
| crossq\_{env}                 | All 11 envs                                    | Per-env tuned (critic width, actor LN)             |

### Running Benchmarks

**Reproduce**: Copy `SPEC_NAME` and `MAX_FRAME` from the table below.

```bash
# PPO: env and max_frame are parameterized via -s flags
source .env && slm-lab run-remote --gpu -s env=ENV -s max_frame=MAX_FRAME \
  slm_lab/spec/benchmark_arc/ppo/ppo_mujoco_arc.yaml SPEC_NAME train -n NAME

# SAC: env and max_frame are hardcoded per spec — no -s flags needed
source .env && slm-lab run-remote --gpu \
  slm_lab/spec/benchmark_arc/sac/sac_mujoco_arc.yaml SPEC_NAME train -n NAME

# CrossQ: env and max_frame are hardcoded per spec — no -s flags needed
source .env && slm-lab run-remote --gpu \
  slm_lab/spec/benchmark/crossq/crossq_mujoco.yaml SPEC_NAME train -n NAME
```

| ENV                       | SPEC\_NAME                           | MAX\_FRAME |
| ------------------------- | ------------------------------------ | ---------- |
| Ant-v5                    | ppo\_ant\_arc                        | 10e6       |
|                           | sac\_ant\_arc                        | 2e6        |
|                           | crossq\_ant                          | 3e6        |
| HalfCheetah-v5            | ppo\_mujoco\_arc                     | 10e6       |
|                           | sac\_halfcheetah\_arc                | 4e6        |
| Hopper-v5                 | ppo\_hopper\_arc                     | 4e6        |
|                           | sac\_hopper\_arc                     | 3e6        |
| Humanoid-v5               | ppo\_mujoco\_arc                     | 10e6       |
|                           | sac\_humanoid\_arc                   | 1e6        |
| HumanoidStandup-v5        | ppo\_mujoco\_arc                     | 4e6        |
|                           | sac\_humanoid\_standup\_arc          | 1e6        |
| InvertedDoublePendulum-v5 | ppo\_inverted\_double\_pendulum\_arc | 10e6       |
|                           | sac\_inverted\_double\_pendulum\_arc | 2e6        |
| InvertedPendulum-v5       | ppo\_inverted\_pendulum\_arc         | 4e6        |
|                           | sac\_inverted\_pendulum\_arc         | 2e6        |
| Pusher-v5                 | ppo\_mujoco\_longhorizon\_arc        | 4e6        |
|                           | sac\_pusher\_arc                     | 1e6        |
| Reacher-v5                | ppo\_mujoco\_longhorizon\_arc        | 4e6        |
|                           | sac\_reacher\_arc                    | 1e6        |
| Swimmer-v5                | ppo\_swimmer\_arc                    | 4e6        |
|                           | sac\_swimmer\_arc                    | 2e6        |
| Walker2d-v5               | ppo\_mujoco\_arc                     | 10e6       |
|                           | sac\_walker2d\_arc                   | 3e6        |

Remote setup: `cp .env.example .env` then set `HF_TOKEN`. See [Remote Training](/slm-lab/using-slm-lab/remote-training.md) for dstack config.

{% hint style="warning" %}
**GPU strongly recommended for MuJoCo.** These benchmarks run 4M-10M frames and take 1-4 hours on cloud GPU (L4/A10G). Local CPU training is not practical. Cloud GPUs via dstack are faster and often cheaper than running on local hardware.
{% endhint %}

### Download and Replay

```bash
# List all available experiments (requires HF_REPO=SLM-Lab/benchmark in .env)
source .env && slm-lab list

# Download a specific experiment
source .env && slm-lab pull ppo_ant_arc

# Replay the trained agent
slm-lab run slm_lab/spec/benchmark_arc/ppo/ppo_mujoco_arc.yaml ppo_ant_arc enjoy@data/ppo_ant_arc_ant_2026_02_12_190644/ppo_ant_arc_t0_spec.json
```

***

## Results

### Ant-v5

[Docs](https://gymnasium.farama.org/environments/mujoco/ant/) | State: Box(105) | Action: Box(8) | Target: >2000

**Settings**: max\_frame 10e6 | num\_envs 16 | max\_session 4 | log\_frequency 1e4

| Algorithm | Status | MA      | SPEC\_FILE                                                                                                                                                | SPEC\_NAME    | HF Data                                                                                                                                        |
| --------- | ------ | ------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- |
| PPO       | ✅      | 2138.28 | [slm\_lab/spec/benchmark\_arc/ppo/ppo\_mujoco\_arc.yaml](https://github.com/kengz/SLM-Lab/blob/master/slm_lab/spec/benchmark_arc/ppo/ppo_mujoco_arc.yaml) | ppo\_ant\_arc | [ppo\_ant\_arc\_ant\_2026\_02\_12\_190644](https://huggingface.co/datasets/SLM-Lab/benchmark/tree/main/data/ppo_ant_arc_ant_2026_02_12_190644) |
| SAC       | ✅      | 4942.91 | [slm\_lab/spec/benchmark\_arc/sac/sac\_mujoco\_arc.yaml](https://github.com/kengz/SLM-Lab/blob/master/slm_lab/spec/benchmark_arc/sac/sac_mujoco_arc.yaml) | sac\_ant\_arc | [sac\_ant\_arc\_2026\_02\_11\_225529](https://huggingface.co/datasets/SLM-Lab/benchmark/tree/main/data/sac_ant_arc_2026_02_11_225529)          |
| CrossQ    | ✅      | 4517.00 | [slm\_lab/spec/benchmark/crossq/crossq\_mujoco.yaml](https://github.com/kengz/SLM-Lab/blob/master/slm_lab/spec/benchmark/crossq/crossq_mujoco.yaml)       | crossq\_ant   | [crossq\_ant\_2026\_03\_01\_102428](https://huggingface.co/datasets/SLM-Lab/benchmark/tree/main/data/crossq_ant_2026_03_01_102428)             |

![Ant-v5](https://huggingface.co/datasets/SLM-Lab/benchmark/resolve/v5.2.0/docs/plots/Ant-v5_multi_trial_graph_mean_returns_ma_vs_frames.png)

### HalfCheetah-v5

[Docs](https://gymnasium.farama.org/environments/mujoco/half_cheetah/) | State: Box(17) | Action: Box(6) | Target: >5000

**Settings**: max\_frame 10e6 | num\_envs 16 | max\_session 4 | log\_frequency 1e4

| Algorithm | Status | MA      | SPEC\_FILE                                                                                                                                                | SPEC\_NAME            | HF Data                                                                                                                                                              |
| --------- | ------ | ------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| PPO       | ✅      | 6240.68 | [slm\_lab/spec/benchmark\_arc/ppo/ppo\_mujoco\_arc.yaml](https://github.com/kengz/SLM-Lab/blob/master/slm_lab/spec/benchmark_arc/ppo/ppo_mujoco_arc.yaml) | ppo\_mujoco\_arc      | [ppo\_mujoco\_arc\_halfcheetah\_2026\_02\_12\_195553](https://huggingface.co/datasets/SLM-Lab/benchmark/tree/main/data/ppo_mujoco_arc_halfcheetah_2026_02_12_195553) |
| SAC       | ✅      | 9815.16 | [slm\_lab/spec/benchmark\_arc/sac/sac\_mujoco\_arc.yaml](https://github.com/kengz/SLM-Lab/blob/master/slm_lab/spec/benchmark_arc/sac/sac_mujoco_arc.yaml) | sac\_halfcheetah\_arc | [sac\_halfcheetah\_4m\_i2\_arc\_2026\_02\_14\_185522](https://huggingface.co/datasets/SLM-Lab/benchmark/tree/main/data/sac_halfcheetah_4m_i2_arc_2026_02_14_185522)  |
| CrossQ    | ✅      | 8616.52 | [slm\_lab/spec/benchmark/crossq/crossq\_mujoco.yaml](https://github.com/kengz/SLM-Lab/blob/master/slm_lab/spec/benchmark/crossq/crossq_mujoco.yaml)       | crossq\_halfcheetah   | [crossq\_halfcheetah\_2026\_03\_01\_101317](https://huggingface.co/datasets/SLM-Lab/benchmark/tree/main/data/crossq_halfcheetah_2026_03_01_101317)                   |

![HalfCheetah-v5](https://huggingface.co/datasets/SLM-Lab/benchmark/resolve/v5.2.0/docs/plots/HalfCheetah-v5_multi_trial_graph_mean_returns_ma_vs_frames.png)

### Hopper-v5

[Docs](https://gymnasium.farama.org/environments/mujoco/hopper/) | State: Box(11) | Action: Box(3) | Target: \~2000

**Settings**: max\_frame 4e6 | num\_envs 16 | max\_session 4 | log\_frequency 1e4

| Algorithm | Status | MA      | SPEC\_FILE                                                                                                                                                | SPEC\_NAME       | HF Data                                                                                                                                                    |
| --------- | ------ | ------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------- |
| PPO       | ⚠️     | 1653.74 | [slm\_lab/spec/benchmark\_arc/ppo/ppo\_mujoco\_arc.yaml](https://github.com/kengz/SLM-Lab/blob/master/slm_lab/spec/benchmark_arc/ppo/ppo_mujoco_arc.yaml) | ppo\_hopper\_arc | [ppo\_hopper\_arc\_hopper\_2026\_02\_12\_222206](https://huggingface.co/datasets/SLM-Lab/benchmark/tree/main/data/ppo_hopper_arc_hopper_2026_02_12_222206) |
| SAC       | ⚠️     | 1416.52 | [slm\_lab/spec/benchmark\_arc/sac/sac\_mujoco\_arc.yaml](https://github.com/kengz/SLM-Lab/blob/master/slm_lab/spec/benchmark_arc/sac/sac_mujoco_arc.yaml) | sac\_hopper\_arc | [sac\_hopper\_3m\_i4\_arc\_2026\_02\_14\_185434](https://huggingface.co/datasets/SLM-Lab/benchmark/tree/main/data/sac_hopper_3m_i4_arc_2026_02_14_185434)  |
| CrossQ    | ⚠️     | 1168.53 | [slm\_lab/spec/benchmark/crossq/crossq\_mujoco.yaml](https://github.com/kengz/SLM-Lab/blob/master/slm_lab/spec/benchmark/crossq/crossq_mujoco.yaml)       | crossq\_hopper   | [crossq\_hopper\_2026\_02\_21\_101148](https://huggingface.co/datasets/SLM-Lab/benchmark/tree/main/data/crossq_hopper_2026_02_21_101148)                   |

![Hopper-v5](https://huggingface.co/datasets/SLM-Lab/benchmark/resolve/v5.2.0/docs/plots/Hopper-v5_multi_trial_graph_mean_returns_ma_vs_frames.png)

### Humanoid-v5

[Docs](https://gymnasium.farama.org/environments/mujoco/humanoid/) | State: Box(348) | Action: Box(17) | Target: >1000

**Settings**: max\_frame 10e6 | num\_envs 16 | max\_session 4 | log\_frequency 1e4

| Algorithm | Status | MA      | SPEC\_FILE                                                                                                                                                | SPEC\_NAME         | HF Data                                                                                                                                                        |
| --------- | ------ | ------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| PPO       | ✅      | 2661.26 | [slm\_lab/spec/benchmark\_arc/ppo/ppo\_mujoco\_arc.yaml](https://github.com/kengz/SLM-Lab/blob/master/slm_lab/spec/benchmark_arc/ppo/ppo_mujoco_arc.yaml) | ppo\_mujoco\_arc   | [ppo\_mujoco\_arc\_humanoid\_2026\_02\_12\_185439](https://huggingface.co/datasets/SLM-Lab/benchmark/tree/main/data/ppo_mujoco_arc_humanoid_2026_02_12_185439) |
| SAC       | ✅      | 1989.65 | [slm\_lab/spec/benchmark\_arc/sac/sac\_mujoco\_arc.yaml](https://github.com/kengz/SLM-Lab/blob/master/slm_lab/spec/benchmark_arc/sac/sac_mujoco_arc.yaml) | sac\_humanoid\_arc | [sac\_humanoid\_arc\_2026\_02\_12\_020016](https://huggingface.co/datasets/SLM-Lab/benchmark/tree/main/data/sac_humanoid_arc_2026_02_12_020016)                |
| CrossQ    | ✅      | 1755.29 | [slm\_lab/spec/benchmark/crossq/crossq\_mujoco.yaml](https://github.com/kengz/SLM-Lab/blob/master/slm_lab/spec/benchmark/crossq/crossq_mujoco.yaml)       | crossq\_humanoid   | [crossq\_humanoid\_2026\_03\_01\_165208](https://huggingface.co/datasets/SLM-Lab/benchmark/tree/main/data/crossq_humanoid_2026_03_01_165208)                   |

![Humanoid-v5](https://huggingface.co/datasets/SLM-Lab/benchmark/resolve/v5.2.0/docs/plots/Humanoid-v5_multi_trial_graph_mean_returns_ma_vs_frames.png)

### HumanoidStandup-v5

[Docs](https://gymnasium.farama.org/environments/mujoco/humanoid_standup/) | State: Box(348) | Action: Box(17) | Target: >100k

**Settings**: max\_frame 4e6 | num\_envs 16 | max\_session 4 | log\_frequency 1e4

| Algorithm | Status | MA        | SPEC\_FILE                                                                                                                                                | SPEC\_NAME                  | HF Data                                                                                                                                                                      |
| --------- | ------ | --------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| PPO       | ✅      | 150104.59 | [slm\_lab/spec/benchmark\_arc/ppo/ppo\_mujoco\_arc.yaml](https://github.com/kengz/SLM-Lab/blob/master/slm_lab/spec/benchmark_arc/ppo/ppo_mujoco_arc.yaml) | ppo\_mujoco\_arc            | [ppo\_mujoco\_arc\_humanoidstandup\_2026\_02\_12\_115050](https://huggingface.co/datasets/SLM-Lab/benchmark/tree/main/data/ppo_mujoco_arc_humanoidstandup_2026_02_12_115050) |
| SAC       | ✅      | 137357.00 | [slm\_lab/spec/benchmark\_arc/sac/sac\_mujoco\_arc.yaml](https://github.com/kengz/SLM-Lab/blob/master/slm_lab/spec/benchmark_arc/sac/sac_mujoco_arc.yaml) | sac\_humanoid\_standup\_arc | [sac\_humanoid\_standup\_arc\_2026\_02\_12\_225150](https://huggingface.co/datasets/SLM-Lab/benchmark/tree/main/data/sac_humanoid_standup_arc_2026_02_12_225150)             |
| CrossQ    | ✅      | 150912.66 | [slm\_lab/spec/benchmark/crossq/crossq\_mujoco.yaml](https://github.com/kengz/SLM-Lab/blob/master/slm_lab/spec/benchmark/crossq/crossq_mujoco.yaml)       | crossq\_humanoid\_standup   | [crossq\_humanoid\_standup\_2026\_02\_28\_184305](https://huggingface.co/datasets/SLM-Lab/benchmark/tree/main/data/crossq_humanoid_standup_2026_02_28_184305)                |

![HumanoidStandup-v5](https://huggingface.co/datasets/SLM-Lab/benchmark/resolve/v5.2.0/docs/plots/HumanoidStandup-v5_multi_trial_graph_mean_returns_ma_vs_frames.png)

### InvertedDoublePendulum-v5

[Docs](https://gymnasium.farama.org/environments/mujoco/inverted_double_pendulum/) | State: Box(9) | Action: Box(1) | Target: \~8000

**Settings**: max\_frame 10e6 | num\_envs 16 | max\_session 4 | log\_frequency 1e4

| Algorithm | Status | MA      | SPEC\_FILE                                                                                                                                                | SPEC\_NAME                           | HF Data                                                                                                                                                                                                                          |
| --------- | ------ | ------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| PPO       | ✅      | 8383.76 | [slm\_lab/spec/benchmark\_arc/ppo/ppo\_mujoco\_arc.yaml](https://github.com/kengz/SLM-Lab/blob/master/slm_lab/spec/benchmark_arc/ppo/ppo_mujoco_arc.yaml) | ppo\_inverted\_double\_pendulum\_arc | [ppo\_inverted\_double\_pendulum\_arc\_inverteddoublependulum\_2026\_02\_12\_225231](https://huggingface.co/datasets/SLM-Lab/benchmark/tree/main/data/ppo_inverted_double_pendulum_arc_inverteddoublependulum_2026_02_12_225231) |
| SAC       | ✅      | 9032.67 | [slm\_lab/spec/benchmark\_arc/sac/sac\_mujoco\_arc.yaml](https://github.com/kengz/SLM-Lab/blob/master/slm_lab/spec/benchmark_arc/sac/sac_mujoco_arc.yaml) | sac\_inverted\_double\_pendulum\_arc | [sac\_inverted\_double\_pendulum\_arc\_2026\_02\_12\_025206](https://huggingface.co/datasets/SLM-Lab/benchmark/tree/main/data/sac_inverted_double_pendulum_arc_2026_02_12_025206)                                                |
| CrossQ    | ✅      | 8027.38 | [slm\_lab/spec/benchmark/crossq/crossq\_mujoco.yaml](https://github.com/kengz/SLM-Lab/blob/master/slm_lab/spec/benchmark/crossq/crossq_mujoco.yaml)       | crossq\_inverted\_double\_pendulum   | [crossq\_inverted\_double\_pendulum\_2026\_03\_01\_101354](https://huggingface.co/datasets/SLM-Lab/benchmark/tree/main/data/crossq_inverted_double_pendulum_2026_03_01_101354)                                                   |

![InvertedDoublePendulum-v5](https://huggingface.co/datasets/SLM-Lab/benchmark/resolve/v5.2.0/docs/plots/InvertedDoublePendulum-v5_multi_trial_graph_mean_returns_ma_vs_frames.png)

### InvertedPendulum-v5

[Docs](https://gymnasium.farama.org/environments/mujoco/inverted_pendulum/) | State: Box(4) | Action: Box(1) | Target: \~1000

**Settings**: max\_frame 4e6 | num\_envs 16 | max\_session 4 | log\_frequency 1e4

| Algorithm | Status | MA     | SPEC\_FILE                                                                                                                                                | SPEC\_NAME                   | HF Data                                                                                                                                                                                               |
| --------- | ------ | ------ | --------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| PPO       | ✅      | 949.94 | [slm\_lab/spec/benchmark\_arc/ppo/ppo\_mujoco\_arc.yaml](https://github.com/kengz/SLM-Lab/blob/master/slm_lab/spec/benchmark_arc/ppo/ppo_mujoco_arc.yaml) | ppo\_inverted\_pendulum\_arc | [ppo\_inverted\_pendulum\_arc\_invertedpendulum\_2026\_02\_12\_062037](https://huggingface.co/datasets/SLM-Lab/benchmark/tree/main/data/ppo_inverted_pendulum_arc_invertedpendulum_2026_02_12_062037) |
| SAC       | ✅      | 928.43 | [slm\_lab/spec/benchmark\_arc/sac/sac\_mujoco\_arc.yaml](https://github.com/kengz/SLM-Lab/blob/master/slm_lab/spec/benchmark_arc/sac/sac_mujoco_arc.yaml) | sac\_inverted\_pendulum\_arc | [sac\_inverted\_pendulum\_arc\_2026\_02\_12\_225503](https://huggingface.co/datasets/SLM-Lab/benchmark/tree/main/data/sac_inverted_pendulum_arc_2026_02_12_225503)                                    |
| CrossQ    | ⚠️     | 877.83 | [slm\_lab/spec/benchmark/crossq/crossq\_mujoco.yaml](https://github.com/kengz/SLM-Lab/blob/master/slm_lab/spec/benchmark/crossq/crossq_mujoco.yaml)       | crossq\_inverted\_pendulum   | [crossq\_inverted\_pendulum\_2026\_02\_28\_184348](https://huggingface.co/datasets/SLM-Lab/benchmark/tree/main/data/crossq_inverted_pendulum_2026_02_28_184348)                                       |

![InvertedPendulum-v5](https://huggingface.co/datasets/SLM-Lab/benchmark/resolve/v5.2.0/docs/plots/InvertedPendulum-v5_multi_trial_graph_mean_returns_ma_vs_frames.png)

### Pusher-v5

[Docs](https://gymnasium.farama.org/environments/mujoco/pusher/) | State: Box(23) | Action: Box(7) | Target: >-50

**Settings**: max\_frame 4e6 | num\_envs 16 | max\_session 4 | log\_frequency 1e4

| Algorithm | Status | MA     | SPEC\_FILE                                                                                                                                                | SPEC\_NAME                    | HF Data                                                                                                                                                                             |
| --------- | ------ | ------ | --------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| PPO       | ✅      | -49.59 | [slm\_lab/spec/benchmark\_arc/ppo/ppo\_mujoco\_arc.yaml](https://github.com/kengz/SLM-Lab/blob/master/slm_lab/spec/benchmark_arc/ppo/ppo_mujoco_arc.yaml) | ppo\_mujoco\_longhorizon\_arc | [ppo\_mujoco\_longhorizon\_arc\_pusher\_2026\_02\_12\_222228](https://huggingface.co/datasets/SLM-Lab/benchmark/tree/main/data/ppo_mujoco_longhorizon_arc_pusher_2026_02_12_222228) |
| SAC       | ✅      | -43.00 | [slm\_lab/spec/benchmark\_arc/sac/sac\_mujoco\_arc.yaml](https://github.com/kengz/SLM-Lab/blob/master/slm_lab/spec/benchmark_arc/sac/sac_mujoco_arc.yaml) | sac\_pusher\_arc              | [sac\_pusher\_arc\_2026\_02\_12\_053603](https://huggingface.co/datasets/SLM-Lab/benchmark/tree/main/data/sac_pusher_arc_2026_02_12_053603)                                         |
| CrossQ    | ✅      | -37.08 | [slm\_lab/spec/benchmark/crossq/crossq\_mujoco.yaml](https://github.com/kengz/SLM-Lab/blob/master/slm_lab/spec/benchmark/crossq/crossq_mujoco.yaml)       | crossq\_pusher                | [crossq\_pusher\_2026\_02\_21\_134637](https://huggingface.co/datasets/SLM-Lab/benchmark/tree/main/data/crossq_pusher_2026_02_21_134637)                                            |

![Pusher-v5](https://huggingface.co/datasets/SLM-Lab/benchmark/resolve/v5.2.0/docs/plots/Pusher-v5_multi_trial_graph_mean_returns_ma_vs_frames.png)

### Reacher-v5

[Docs](https://gymnasium.farama.org/environments/mujoco/reacher/) | State: Box(10) | Action: Box(2) | Target: >-10

**Settings**: max\_frame 4e6 | num\_envs 16 | max\_session 4 | log\_frequency 1e4

| Algorithm | Status | MA    | SPEC\_FILE                                                                                                                                                | SPEC\_NAME                    | HF Data                                                                                                                                                                               |
| --------- | ------ | ----- | --------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| PPO       | ✅      | -5.03 | [slm\_lab/spec/benchmark\_arc/ppo/ppo\_mujoco\_arc.yaml](https://github.com/kengz/SLM-Lab/blob/master/slm_lab/spec/benchmark_arc/ppo/ppo_mujoco_arc.yaml) | ppo\_mujoco\_longhorizon\_arc | [ppo\_mujoco\_longhorizon\_arc\_reacher\_2026\_02\_12\_115033](https://huggingface.co/datasets/SLM-Lab/benchmark/tree/main/data/ppo_mujoco_longhorizon_arc_reacher_2026_02_12_115033) |
| SAC       | ✅      | -6.31 | [slm\_lab/spec/benchmark\_arc/sac/sac\_mujoco\_arc.yaml](https://github.com/kengz/SLM-Lab/blob/master/slm_lab/spec/benchmark_arc/sac/sac_mujoco_arc.yaml) | sac\_reacher\_arc             | [sac\_reacher\_arc\_2026\_02\_12\_055200](https://huggingface.co/datasets/SLM-Lab/benchmark/tree/main/data/sac_reacher_arc_2026_02_12_055200)                                         |
| CrossQ    | ✅      | -5.65 | [slm\_lab/spec/benchmark/crossq/crossq\_mujoco.yaml](https://github.com/kengz/SLM-Lab/blob/master/slm_lab/spec/benchmark/crossq/crossq_mujoco.yaml)       | crossq\_reacher               | [crossq\_reacher\_2026\_02\_28\_184304](https://huggingface.co/datasets/SLM-Lab/benchmark/tree/main/data/crossq_reacher_2026_02_28_184304)                                            |

![Reacher-v5](https://huggingface.co/datasets/SLM-Lab/benchmark/resolve/v5.2.0/docs/plots/Reacher-v5_multi_trial_graph_mean_returns_ma_vs_frames.png)

### Swimmer-v5

[Docs](https://gymnasium.farama.org/environments/mujoco/swimmer/) | State: Box(8) | Action: Box(2) | Target: >200

**Settings**: max\_frame 4e6 | num\_envs 16 | max\_session 4 | log\_frequency 1e4

| Algorithm | Status | MA     | SPEC\_FILE                                                                                                                                                | SPEC\_NAME        | HF Data                                                                                                                                                        |
| --------- | ------ | ------ | --------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| PPO       | ✅      | 282.44 | [slm\_lab/spec/benchmark\_arc/ppo/ppo\_mujoco\_arc.yaml](https://github.com/kengz/SLM-Lab/blob/master/slm_lab/spec/benchmark_arc/ppo/ppo_mujoco_arc.yaml) | ppo\_swimmer\_arc | [ppo\_swimmer\_arc\_swimmer\_2026\_02\_12\_100445](https://huggingface.co/datasets/SLM-Lab/benchmark/tree/main/data/ppo_swimmer_arc_swimmer_2026_02_12_100445) |
| SAC       | ✅      | 301.34 | [slm\_lab/spec/benchmark\_arc/sac/sac\_mujoco\_arc.yaml](https://github.com/kengz/SLM-Lab/blob/master/slm_lab/spec/benchmark_arc/sac/sac_mujoco_arc.yaml) | sac\_swimmer\_arc | [sac\_swimmer\_arc\_2026\_02\_12\_054349](https://huggingface.co/datasets/SLM-Lab/benchmark/tree/main/data/sac_swimmer_arc_2026_02_12_054349)                  |
| CrossQ    | ✅      | 221.12 | [slm\_lab/spec/benchmark/crossq/crossq\_mujoco.yaml](https://github.com/kengz/SLM-Lab/blob/master/slm_lab/spec/benchmark/crossq/crossq_mujoco.yaml)       | crossq\_swimmer   | [crossq\_swimmer\_2026\_02\_21\_184204](https://huggingface.co/datasets/SLM-Lab/benchmark/tree/main/data/crossq_swimmer_2026_02_21_184204)                     |

![Swimmer-v5](https://huggingface.co/datasets/SLM-Lab/benchmark/resolve/v5.2.0/docs/plots/Swimmer-v5_multi_trial_graph_mean_returns_ma_vs_frames.png)

### Walker2d-v5

[Docs](https://gymnasium.farama.org/environments/mujoco/walker2d/) | State: Box(17) | Action: Box(6) | Target: >3500

**Settings**: max\_frame 10e6 | num\_envs 16 | max\_session 4 | log\_frequency 1e4

| Algorithm | Status | MA      | SPEC\_FILE                                                                                                                                                | SPEC\_NAME         | HF Data                                                                                                                                                        |
| --------- | ------ | ------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| PPO       | ✅      | 4378.62 | [slm\_lab/spec/benchmark\_arc/ppo/ppo\_mujoco\_arc.yaml](https://github.com/kengz/SLM-Lab/blob/master/slm_lab/spec/benchmark_arc/ppo/ppo_mujoco_arc.yaml) | ppo\_mujoco\_arc   | [ppo\_mujoco\_arc\_walker2d\_2026\_02\_12\_190312](https://huggingface.co/datasets/SLM-Lab/benchmark/tree/main/data/ppo_mujoco_arc_walker2d_2026_02_12_190312) |
| SAC       | ⚠️     | 3123.66 | [slm\_lab/spec/benchmark\_arc/sac/sac\_mujoco\_arc.yaml](https://github.com/kengz/SLM-Lab/blob/master/slm_lab/spec/benchmark_arc/sac/sac_mujoco_arc.yaml) | sac\_walker2d\_arc | [sac\_walker2d\_3m\_i4\_arc\_2026\_02\_14\_185550](https://huggingface.co/datasets/SLM-Lab/benchmark/tree/main/data/sac_walker2d_3m_i4_arc_2026_02_14_185550)  |
| CrossQ    | ✅      | 4389.62 | [slm\_lab/spec/benchmark/crossq/crossq\_mujoco.yaml](https://github.com/kengz/SLM-Lab/blob/master/slm_lab/spec/benchmark/crossq/crossq_mujoco.yaml)       | crossq\_walker2d   | [crossq\_walker2d\_2026\_02\_28\_184343](https://huggingface.co/datasets/SLM-Lab/benchmark/tree/main/data/crossq_walker2d_2026_02_28_184343)                   |

![Walker2d-v5](https://huggingface.co/datasets/SLM-Lab/benchmark/resolve/v5.2.0/docs/plots/Walker2d-v5_multi_trial_graph_mean_returns_ma_vs_frames.png)

**Legend:** ✅ Solved | ⚠️ Close (>80%) | ❌ Failed

***

## CrossQ Wall-Clock Speedup vs SAC

CrossQ eliminates target networks via cross batch normalization, enabling UTD=1 at \~700 fps — 3.5–6.7x faster than SAC on the same hardware.

| Env                | CrossQ FPS | SAC FPS | Speedup |
| ------------------ | ---------- | ------- | ------- |
| HalfCheetah-v5     | 705        | 200     | 3.5x    |
| Hopper-v5          | 693        | 104     | 6.7x    |
| Walker2d-v5        | \~700      | 104     | 6.7x    |
| Ant-v5             | \~700      | 200     | 3.5x    |
| Humanoid-v5        | \~350      | 53      | 6.6x    |
| HumanoidStandup-v5 | 340        | 53      | 6.4x    |

> Measured on RTX 3090. CrossQ achieves comparable scores at significantly lower wall-clock time.

***

## Historical Results (v4)

<details>

<summary>Roboschool Results (v4) - click to expand</summary>

{% hint style="warning" %}
**Deprecated:** Roboschool is abandoned (MuJoCo became free in 2022). These v4 results are preserved for historical reference only. Use Gymnasium MuJoCo environments for new work.

Environment mapping: `RoboschoolHopper-v1` → `Hopper-v5`, `RoboschoolHalfCheetah-v1` → `HalfCheetah-v5`, etc.
{% endhint %}

* [Upload PR #427](https://github.com/kengz/SLM-Lab/pull/427)
* [Google Drive data](https://drive.google.com/file/d/1_rVeXPuZoifXuJmyzY8vC5yregH4DTXo/view?usp=sharing)

| Env. \ Alg.                      | A2C (GAE) | A2C (n-step) | PPO   | SAC        |
| -------------------------------- | --------- | ------------ | ----- | ---------- |
| RoboschoolAnt                    | 787       | 1396         | 1843  | **2915**   |
| RoboschoolHalfCheetah            | 712       | 439          | 1960  | **2497**   |
| RoboschoolHopper                 | 710       | 285          | 2042  | **2045**   |
| RoboschoolInvertedDoublePendulum | 996       | 4410         | 8076  | **8085**   |
| RoboschoolInvertedPendulum       | **995**   | 978          | 986   | 941        |
| RoboschoolReacher                | 12.9      | 10.16        | 19.51 | **19.99**  |
| RoboschoolWalker2d               | 280       | 220          | 1660  | **1894**   |
| RoboschoolHumanoid               | 99.31     | 54.58        | 2388  | **2621**\* |

> Episode score at the end of training. Reported scores are the average over the last 100 checkpoints, averaged over 4 Sessions. Results marked with `*` required 50M-100M frames using async SAC.

</details>