🎯Discrete Benchmark

Classic Control & Box2D Results

SLM Lab v5.2 validates algorithms on Gymnasium discrete environments using the TorchArc architecture. These benchmarks cover:

• Classic Control: CartPole, Acrobot, Pendulum—simple physics tasks ideal for algorithm validation

• Box2D: LunarLander—2D physics with more complex dynamics

Results below are from February–March 2026 benchmark runs using Gymnasium v5 environments. TorchArc specs for existing algorithms; CrossQ uses standard MLP specs.

All trained models and metrics are publicly available on HuggingFace.

Methodology

Results show Trial-level performance:

1. Trial = 4 Sessions with different random seeds

2. Session = One complete training run

3. Score = Final 100-checkpoint moving average (total_reward_ma)

The trial score is the mean across 4 sessions, providing statistically meaningful results.

Standardized Settings

Category	num_envs	max_frame	log_frequency	ASHA grace_period
Classic Control	4	2e5-3e5	500	1e4
Box2D	8	3e5	1000	5e4

The grace_period is the minimum frames before ASHA early stopping can terminate underperforming trials.

v5 vs v4 Difficulty: Gymnasium environments have stricter termination and reward handling:

• LunarLander-v3 is notably harder than v2—stricter landing criteria, lower typical scores

• Pendulum-v1 uses different reward scaling than v0

• Expect 5-15% lower scores compared to OpenAI Gym benchmarks

See Gymnasium docs for environment-specific changes.

Running Benchmarks

Local - runs on your machine (Classic Control completes in minutes on CPU):

slm-lab run slm_lab/spec/benchmark_arc/ppo/ppo_classic_arc.yaml ppo_cartpole_arc train
slm-lab run slm_lab/spec/benchmark_arc/dqn/dqn_box2d_arc.yaml ddqn_per_concat_lunar_arc train

Remote - cloud GPU via dstack, auto-syncs to HuggingFace:

source .env && slm-lab run-remote slm_lab/spec/benchmark_arc/ppo/ppo_classic_arc.yaml ppo_cartpole_arc train -n cartpole
source .env && slm-lab run-remote slm_lab/spec/benchmark_arc/dqn/dqn_box2d_arc.yaml ddqn_per_concat_lunar_arc train -n lunar

Remote setup: cp .env.example .env then set HF_TOKEN. See Remote Training for dstack config.

GPU not required for Classic Control. These environments train fast on CPU. Box2D (LunarLander) benefits from GPU but still runs fine locally.

Download and Replay

# List all available experiments (requires HF_REPO=SLM-Lab/benchmark in .env)
source .env && slm-lab list

# Download a specific experiment
source .env && slm-lab pull ppo_cartpole_arc

# Replay the trained agent
slm-lab run slm_lab/spec/benchmark_arc/ppo/ppo_classic_arc.yaml ppo_cartpole_arc enjoy@data/ppo_cartpole_arc_*/ppo_cartpole_arc_t0_spec.yaml

---

Results

Classic Control

CartPole-v1

Docs | State: Box(4) | Action: Discrete(2) | Target: >400

Settings: max_frame 2e5 | num_envs 4 | max_session 4 | log_frequency 500

Algorithm	Status	MA	SPEC_FILE	SPEC_NAME	HF Data
REINFORCE	✅	483.31	slm_lab/spec/benchmark_arc/reinforce/reinforce_arc.yaml	reinforce_cartpole_arc	reinforce_cartpole_arc_2026_02_11_135616
SARSA	✅	430.95	slm_lab/spec/benchmark_arc/sarsa/sarsa_arc.yaml	sarsa_boltzmann_cartpole_arc	sarsa_boltzmann_cartpole_arc_2026_02_11_135616
DQN	⚠️	239.94	slm_lab/spec/benchmark_arc/dqn/dqn_classic_arc.yaml	dqn_boltzmann_cartpole_arc	dqn_boltzmann_cartpole_arc_2026_02_11_135648
DDQN+PER	✅	451.51	slm_lab/spec/benchmark_arc/dqn/dqn_classic_arc.yaml	ddqn_per_boltzmann_cartpole_arc	ddqn_per_boltzmann_cartpole_arc_2026_02_11_140518
A2C	✅	496.68	slm_lab/spec/benchmark_arc/a2c/a2c_classic_arc.yaml	a2c_gae_cartpole_arc	a2c_gae_cartpole_arc_2026_02_11_142531
PPO	✅	498.94	slm_lab/spec/benchmark_arc/ppo/ppo_classic_arc.yaml	ppo_cartpole_arc	ppo_cartpole_arc_2026_02_11_144029
SAC	✅	406.09	slm_lab/spec/benchmark_arc/sac/sac_classic_arc.yaml	sac_cartpole_arc	sac_cartpole_arc_2026_02_11_144155
CrossQ	⚠️	334.59	slm_lab/spec/benchmark/crossq/crossq_classic.yaml	crossq_cartpole	crossq_cartpole_2026_03_02_100434

CartPole-v1

Acrobot-v1

Docs | State: Box(6) | Action: Discrete(3) | Target: >-100

Settings: max_frame 3e5 | num_envs 4 | max_session 4 | log_frequency 500

Algorithm	Status	MA	SPEC_FILE	SPEC_NAME	HF Data
DQN	✅	-94.17	slm_lab/spec/benchmark_arc/dqn/dqn_classic_arc.yaml	dqn_boltzmann_acrobot_arc	dqn_boltzmann_acrobot_arc_2026_02_11_144342
DDQN+PER	✅	-83.92	slm_lab/spec/benchmark_arc/dqn/dqn_classic_arc.yaml	ddqn_per_acrobot_arc	ddqn_per_acrobot_arc_2026_02_11_153725
A2C	✅	-83.99	slm_lab/spec/benchmark_arc/a2c/a2c_classic_arc.yaml	a2c_gae_acrobot_arc	a2c_gae_acrobot_arc_2026_02_11_153806
PPO	✅	-81.28	slm_lab/spec/benchmark_arc/ppo/ppo_classic_arc.yaml	ppo_acrobot_arc	ppo_acrobot_arc_2026_02_11_153758
SAC	✅	-92.60	slm_lab/spec/benchmark_arc/sac/sac_classic_arc.yaml	sac_acrobot_arc	sac_acrobot_arc_2026_02_11_162211
CrossQ	✅	-103.13	slm_lab/spec/benchmark/crossq/crossq_classic.yaml	crossq_acrobot	crossq_acrobot_2026_02_23_153622

Acrobot-v1

Pendulum-v1

Docs | State: Box(3) | Action: Box(1) | Target: >-200

Settings: max_frame 3e5 | num_envs 4 | max_session 4 | log_frequency 500

Algorithm	Status	MA	SPEC_FILE	SPEC_NAME	HF Data
A2C	❌	-820.74	slm_lab/spec/benchmark_arc/a2c/a2c_classic_arc.yaml	a2c_gae_pendulum_arc	a2c_gae_pendulum_arc_2026_02_11_162217
PPO	✅	-174.87	slm_lab/spec/benchmark_arc/ppo/ppo_classic_arc.yaml	ppo_pendulum_arc	ppo_pendulum_arc_2026_02_11_162156
SAC	✅	-150.97	slm_lab/spec/benchmark_arc/sac/sac_classic_arc.yaml	sac_pendulum_arc	sac_pendulum_arc_2026_02_11_162240
CrossQ	✅	-145.66	slm_lab/spec/benchmark/crossq/crossq_classic.yaml	crossq_pendulum	crossq_pendulum_2026_02_28_130648

Pendulum-v1

Box2D

LunarLander-v3 (Discrete)

Docs | State: Box(8) | Action: Discrete(4) | Target: >200

Settings: max_frame 3e5 | num_envs 8 | max_session 4 | log_frequency 1000

Algorithm	Status	MA	SPEC_FILE	SPEC_NAME	HF Data
DQN	⚠️	195.21	slm_lab/spec/benchmark_arc/dqn/dqn_box2d_arc.yaml	dqn_concat_lunar_arc	dqn_concat_lunar_arc_2026_02_11_201407
DDQN+PER	✅	265.90	slm_lab/spec/benchmark_arc/dqn/dqn_box2d_arc.yaml	ddqn_per_concat_lunar_arc	ddqn_per_concat_lunar_arc_2026_02_13_105115
A2C	❌	27.38	slm_lab/spec/benchmark_arc/a2c/a2c_classic_arc.yaml	a2c_gae_lunar_arc	a2c_gae_lunar_arc_2026_02_11_224304
PPO	⚠️	183.30	slm_lab/spec/benchmark_arc/ppo/ppo_box2d_arc.yaml	ppo_lunar_arc	ppo_lunar_arc_2026_02_11_201303
SAC	⚠️	106.17	slm_lab/spec/benchmark_arc/sac/sac_box2d_arc.yaml	sac_lunar_arc	sac_lunar_arc_2026_02_11_201417
CrossQ	❌	139.21	slm_lab/spec/benchmark/crossq/crossq_box2d.yaml	crossq_lunar	crossq_lunar_2026_02_28_130733

LunarLander-v3 Discrete

LunarLander-v3 (Continuous)

Docs | State: Box(8) | Action: Box(2) | Target: >200

Settings: max_frame 3e5 | num_envs 8 | max_session 4 | log_frequency 1000

Algorithm	Status	MA	SPEC_FILE	SPEC_NAME	HF Data
A2C	❌	-76.81	slm_lab/spec/benchmark_arc/a2c/a2c_classic_arc.yaml	a2c_gae_lunar_continuous_arc	a2c_gae_lunar_continuous_arc_2026_02_11_224301
PPO	⚠️	132.58	slm_lab/spec/benchmark_arc/ppo/ppo_box2d_arc.yaml	ppo_lunar_continuous_arc	ppo_lunar_continuous_arc_2026_02_11_224229
SAC	⚠️	125.00	slm_lab/spec/benchmark_arc/sac/sac_box2d_arc.yaml	sac_lunar_continuous_arc	sac_lunar_continuous_arc_2026_02_12_222203
CrossQ	✅	268.91	slm_lab/spec/benchmark/crossq/crossq_box2d.yaml	crossq_lunar_continuous	crossq_lunar_continuous_2026_03_01_140517

LunarLander-v3 Continuous

Legend: ✅ Solved | ⚠️ Close (>80%) | ❌ Failed

---

Historical Results (v4)

OpenAI Gym Results (v4) - click to expand

These results from SLM Lab v4 used OpenAI Gym environments (now deprecated). Environment versions differ from current Gymnasium versions. Unity environments are no longer included in the core package.

• Upload PR #427

• Google Drive data

Env. \ Alg.	DQN	DDQN+PER	A2C (GAE)	A2C (n-step)	PPO	SAC
Breakout	80.88	182	377	398	443	3.51*
Pong	18.48	20.5	19.31	19.56	20.58	19.87*
Qbert	5494	11426	12405	13590	13460	923*
Seaquest	1185	4405	1070	1684	1715	171*
LunarLander	192	233	25.21	68.23	214	276

Episode score at the end of training. Reported scores are the average over the last 100 checkpoints, averaged over 4 Sessions. Results marked with * used async SAC.

For the full Atari benchmark, see Atari Benchmark.