👾Atari Benchmark
A2C, PPO & SAC Atari Results
SLM Lab v5.2 validates A2C, PPO, and SAC on ALE (Arcade Learning Environment) environments using the TorchArc neural network architecture. The ALE provides 50+ classic Atari 2600 games as standardized RL benchmarks.
57 games tested with all results available on HuggingFace.
v5 vs v4 Difficulty: Gymnasium ALE v5 is significantly harder than OpenAI Gym's NoFrameskip-v4:
Sticky actions (
repeat_action_probability=0.25) per Machado et al. (2018)Deterministic frame skipping with proper action handling
Stricter termination conditions
Expect 10-40% lower scores compared to older benchmarks. Some games (Bowling, Skiing) are much harder in v5.
Methodology
Results show Trial-level performance:
Trial = 4 Sessions with different random seeds
Session = One complete training run
Score = Final 100-checkpoint moving average (
total_reward_ma)
The trial score is the mean across 4 sessions, providing statistically meaningful results.
Configuration
Settings: num_envs 16 | max_session 4 | log_frequency 10000
Algorithm Specs (all use Nature CNN [32,64,64] + 512fc):
A2C: a2c_atari_arc.yaml - RMSprop (lr=7e-4), training_frequency=32, max_frame=10e6
PPO: ppo_atari_arc.yaml - AdamW (lr=2.5e-4), minibatch=256, horizon=128, epochs=4, max_frame=10e6
SAC: sac_atari_arc.yaml - Categorical SAC, AdamW (lr=3e-4), training_iter=3, training_frequency=4, max_frame=2e6
CrossQ (experimental): crossq_atari.yaml - Categorical CrossQ, AdamW (lr=1e-3), training_iter=3, FC1024 critics, max_frame=2e6 — tested on 6 games only
Environment: Gymnasium ALE v5 with life_loss_info=true, sticky actions (repeat_action_probability=0.25)
PPO Lambda Variants
Different games benefit from different lambda values for GAE. All variants use the same spec file:
ppo_atari_arc
0.95
Strategic games (default)
ppo_atari_lam85_arc
0.85
Mixed games
ppo_atari_lam70_arc
0.70
Action games
Running Benchmarks
All games use the same spec file with variable substitution for the environment.
Remote (recommended) - cloud GPU via dstack, auto-syncs to HuggingFace:
Remote setup: cp .env.example .env then set HF_TOKEN. See Remote Training for dstack config.
Local - runs on your machine (requires GPU, ~2-3 hours per game):
GPU required for Atari. Each game runs 10M frames and takes 2-3 hours on cloud GPU (L4/A10G). Local CPU training is not practical. Cloud GPUs via dstack are faster and often cheaper than running on local hardware.
Download and Replay
Results
Skipped: Adventure, MontezumaRevenge, Pitfall, PrivateEye, Venture (hard exploration), ElevatorAction (deprecated env)
Training Curves
A2C vs PPO vs SAC mean returns (moving average) vs training frames. Shaded regions show standard deviation across 4 sessions.
Historical Results (v4)
Last updated
Was this helpful?