🏃Continuous Benchmark

MuJoCo Benchmark Results (v5)

SLM Lab v5 validates PPO on Gymnasium MuJoCo environments. MuJoCo (Multi-Joint dynamics with Contact) provides physics simulation for continuous control tasks ranging from simple pendulums to complex humanoid locomotion.

Results below are from January 2026 benchmark reruns using MuJoCo v5 environments.

All trained models and metrics are publicly available on HuggingFace.

Methodology

Results show Trial-level performance:

1. Trial = 4 Sessions with different random seeds

2. Session = One complete training run

3. Score = Final 100-checkpoint moving average (total_reward_ma)

The trial score is the mean across 4 sessions, providing statistically meaningful results.

Standardized Settings

Category	num_envs	max_frame	log_frequency	ASHA grace_period
MuJoCo	16	4e6-10e6	10000	1e5-1e6

The grace_period is the minimum frames before ASHA early stopping can terminate underperforming trials.

v5 vs v4 Difficulty: Gymnasium MuJoCo v5 environments are significantly harder than v4. Key changes include:

• Updated physics engine with more accurate contact dynamics

• Revised reward functions with stricter success criteria

• Termination conditions more closely match real-world failure modes

Expect 10-30% lower scores compared to v4 benchmarks. See Gymnasium Migration Guide for details.

Running Benchmarks

Remote (recommended) - cloud GPU via dstack, auto-syncs to HuggingFace:

# Individual spec
source .env && slm-lab run-remote --gpu slm_lab/spec/benchmark/ppo/ppo_hopper.json ppo_hopper train -n hopper

# Unified spec with variable substitution
source .env && slm-lab run-remote --gpu -s env=Humanoid-v5 -s max_frame=10e6 \
  slm_lab/spec/benchmark/ppo/ppo_mujoco.json ppo_mujoco train -n humanoid

Remote setup: cp .env.example .env then set HF_TOKEN. See Remote Training for dstack config.

Local - runs on your machine (requires decent GPU, runs 1-4 hours):

slm-lab run slm_lab/spec/benchmark/ppo/ppo_hopper.json ppo_hopper train
slm-lab run -s env=Humanoid-v5 -s max_frame=10e6 slm_lab/spec/benchmark/ppo/ppo_mujoco.json ppo_mujoco train

GPU strongly recommended for MuJoCo. These benchmarks run 4M-10M frames and take 1-4 hours on cloud GPU (L4/A10G). Local CPU training is not practical. Cloud GPUs via dstack are faster and often cheaper than running on local hardware.

Download and Replay

# List all available experiments (requires HF_REPO=SLM-Lab/benchmark in .env)
source .env && slm-lab list

# Download a specific experiment
source .env && slm-lab pull ppo_hopper

# Replay the trained agent
slm-lab run slm_lab/spec/benchmark/ppo/ppo_hopper.json ppo_hopper enjoy@data/ppo_hopper_2026_01_31_105438/ppo_hopper_t0_spec.json

---

Results

January 2026 Rerun: SAC benchmarks are omitted due to compute constraints (off-policy algorithms require significantly more resources). PPO results cover all 11 MuJoCo environments.

Hopper-v5

Docs | State: Box(11) | Action: Box(3) | Target: ~2000

Settings: max_frame 4e6 | num_envs 16 | max_session 4 | log_frequency 1e4

Algorithm	Status	MA	Spec	HuggingFace
PPO	✅	1972	ppo_hopper.json	ppo_hopper_2026_01_31

Hopper-v5

HalfCheetah-v5

Docs | State: Box(17) | Action: Box(6) | Target: >5000

Settings: max_frame 10e6 | num_envs 16 | max_session 4 | log_frequency 1e4

Algorithm	Status	MA	Spec	HuggingFace
PPO	✅	5852	ppo_mujoco.json	ppo_mujoco_halfcheetah_2026_01_30

HalfCheetah-v5

Walker2d-v5

Docs | State: Box(17) | Action: Box(6) | Target: >3500

Settings: max_frame 10e6 | num_envs 16 | max_session 4 | log_frequency 1e4

Algorithm	Status	MA	Spec	HuggingFace
PPO	✅	4042	ppo_mujoco.json	ppo_mujoco_walker2d_2026_01_30

Walker2d-v5

Ant-v5

Docs | State: Box(105) | Action: Box(8) | Target: >2000

Settings: max_frame 10e6 | num_envs 16 | max_session 4 | log_frequency 1e4

Algorithm	Status	MA	Spec	HuggingFace
PPO	✅	2515	ppo_ant.json	ppo_ant_2026_01_31

Ant-v5

Swimmer-v5

Docs | State: Box(8) | Action: Box(2) | Target: >200

Settings: max_frame 4e6 | num_envs 16 | max_session 4 | log_frequency 1e4

Algorithm	Status	MA	Spec	HuggingFace
PPO	✅	229	ppo_swimmer.json	ppo_swimmer_2026_01_30

Swimmer-v5

Reacher-v5

Docs | State: Box(10) | Action: Box(2) | Target: >-10

Settings: max_frame 4e6 | num_envs 16 | max_session 4 | log_frequency 1e4

Algorithm	Status	MA	Spec	HuggingFace
PPO	✅	-5.08	ppo_mujoco.json	ppo_mujoco_longhorizon_reacher_2026_01_30

Reacher-v5

Pusher-v5

Docs | State: Box(23) | Action: Box(7) | Target: >-50

Settings: max_frame 4e6 | num_envs 16 | max_session 4 | log_frequency 1e4

Algorithm	Status	MA	Spec	HuggingFace
PPO	✅	-49.1	ppo_mujoco.json	ppo_mujoco_longhorizon_pusher_2026_01_30

Pusher-v5

InvertedPendulum-v5

Docs | State: Box(4) | Action: Box(1) | Target: ~1000

Settings: max_frame 4e6 | num_envs 16 | max_session 4 | log_frequency 1e4

Algorithm	Status	MA	Spec	HuggingFace
PPO	✅	945	ppo_inverted_pendulum.json	ppo_inverted_pendulum_2026_01_30

InvertedPendulum-v5

InvertedDoublePendulum-v5

Docs | State: Box(9) | Action: Box(1) | Target: ~8000

Settings: max_frame 10e6 | num_envs 16 | max_session 4 | log_frequency 1e4

Algorithm	Status	MA	Spec	HuggingFace
PPO	✅	7622	ppo_inverted_double_pendulum.json	ppo_inverted_double_pendulum_2026_01_30

InvertedDoublePendulum-v5

Humanoid-v5

Docs | State: Box(348) | Action: Box(17) | Target: >1000

Settings: max_frame 10e6 | num_envs 16 | max_session 4 | log_frequency 1e4

Algorithm	Status	MA	Spec	HuggingFace
PPO	✅	3774	ppo_mujoco.json	ppo_mujoco_humanoid_2026_01_30

Humanoid-v5

HumanoidStandup-v5

Docs | State: Box(348) | Action: Box(17) | Target: >100k

Settings: max_frame 4e6 | num_envs 16 | max_session 4 | log_frequency 1e4

Algorithm	Status	MA	Spec	HuggingFace
PPO	✅	165841	ppo_mujoco.json	ppo_mujoco_humanoidstandup_2026_01_30

HumanoidStandup-v5

Legend: ✅ Solved | ⚠️ Close (>80%) | ❌ Failed

---

Historical Results (v4)

Roboschool Results (v4) - click to expand

Deprecated: Roboschool is abandoned (MuJoCo became free in 2022). These v4 results are preserved for historical reference only. Use Gymnasium MuJoCo environments for new work.

Environment mapping: RoboschoolHopper-v1 → Hopper-v5, RoboschoolHalfCheetah-v1 → HalfCheetah-v5, etc.

• Upload PR #427

• Google Drive data

Env. \ Alg.	A2C (GAE)	A2C (n-step)	PPO	SAC
RoboschoolAnt	787	1396	1843	2915
RoboschoolHalfCheetah	712	439	1960	2497
RoboschoolHopper	710	285	2042	2045
RoboschoolInvertedDoublePendulum	996	4410	8076	8085
RoboschoolInvertedPendulum	995	978	986	941
RoboschoolReacher	12.9	10.16	19.51	19.99
RoboschoolWalker2d	280	220	1660	1894
RoboschoolHumanoid	99.31	54.58	2388	2621*

Episode score at the end of training. Reported scores are the average over the last 100 checkpoints, averaged over 4 Sessions. Results marked with * required 50M-100M frames using async SAC.