📚Deep RL Resources

Master List

For a comprehensive, continuously updated collection:

GitHub - kengz/awesome-deep-rl: A curated list of awesome Deep Reinforcement Learning resources.GitHub

Books

Recommended order for learning:

1. Graesser and Keng, Foundations of Deep Reinforcement Learning

   • Best for beginners; SLM Lab is the companion library

   • Covers REINFORCE, DQN, A2C, PPO with code examples

2. Sutton and Barto, Reinforcement Learning: An Introduction

   • The classic RL textbook; comprehensive theory

   • Free online version

3. Francois-Lavet et al., An Introduction to Deep Reinforcement Learning

   • More technical; good for researchers

   • Free arXiv version

Video Courses

Course	Level	Focus
David Silver UCL RL Course	Intro	Classic RL theory
Deep RL Bootcamp 2017	Intermediate	Practical deep RL
DeepMind UCL Deep RL 2018	Intermediate	Modern deep RL
Sergey Levine CS294	Advanced	Research-level

Tutorials

Getting started:

• Andrew Karpathy: Pong from Pixels - Classic intro, policy gradients from scratch

• OpenAI Spinning Up - Excellent intro with clean implementations

Code-focused:

• dennybritz/reinforcement-learning - Many algorithm implementations

• higgsfield/RL-Adventure - DQN variants

• higgsfield/RL-Adventure-2 - Policy gradient methods

Papers by Algorithm

Value-Based Methods

Algorithm	Paper	Key Idea
DQN	Mnih et al. 2013	Deep Q-learning with experience replay
Double DQN	van Hasselt et al. 2015	Reduce overestimation bias
Dueling DQN	Wang et al. 2016	Separate value and advantage streams
PER	Schaul et al. 2015	Prioritize important transitions
CER	Zhang & Sutton 2017	Always include latest transition

Policy Gradient Methods

Algorithm	Paper	Key Idea
A3C	Mnih et al. 2016	Asynchronous actor-critic
GAE	Schulman et al. 2015	Better advantage estimation
PPO	Schulman et al. 2017	Clipped surrogate objective
SAC	Haarnoja et al. 2018	Maximum entropy RL

Other Notable Papers

Topic	Paper	Relevance
Benchmarking	Henderson et al. 2017	Why implementation details matter
HER	Andrychowicz et al. 2017	Learning from failures
QT-Opt	Kalashnikov et al. 2018	Real-world robot learning
SIL	Oh et al. 2018	Learn from good past experiences

Reference Implementations

Library	Focus	Notes
SLM Lab	Research, modularity	This project
CleanRL	Single-file implementations	Great for learning
Stable Baselines3	Production use	Well-tested, easy API
RLlib	Distributed training	Scalable

Staying Current

• r/reinforcementlearning - Community discussions

• Papers With Code - RL - Benchmarks and papers

• @hardmaru - ML research highlights