SLM Lab
v4.1.1
v4.1.1
  • SLM Lab
  • 🖥Setup
    • Installation
    • Quick Start
  • 🚀Using SLM Lab
    • Lab Command
    • Lab Organization
    • Train and Enjoy: REINFORCE CartPole
    • Agent Spec: DDQN+PER on LunarLander
    • Env Spec: A2C on Pong
    • GPU Usage: PPO on Pong
    • Parallelizing Training: Async SAC on Humanoid
    • Experiment and Search Spec: PPO on Breakout
    • Run Benchmark: A2C on Atari Games
    • Meta Spec: High Level Specifications
    • Post-Hoc Analysis
    • TensorBoard: Visualizing Models and Actions
    • Using SLM Lab In Your Project
  • 📈Analyzing Results
    • Data Locations
    • Graphs and Data
    • Performance Metrics
  • 🥇Benchmark Results
    • Public Benchmark Data
    • Discrete Environment Benchmark
    • Continuous Environment Benchmark
    • Atari Environment Benchmark
    • RL GIFs
  • 🔧Development
    • Modular Design
      • Algorithm Taxonomy
      • Class Inheritance: A2C > PPO
    • Algorithm
      • DQN
      • REINFORCE
      • Actor Critic
    • Memory
      • Replay
      • PrioritizedReplay
      • OnPolicyReplay
      • OnPolicyBatchReplay
    • Net
      • MLP
      • CNN
      • RNN
    • Profiling SLM Lab
  • 📖Publications and Talks
    • Book: Foundations of Deep Reinforcement Learning
    • Talks and Presentations
  • 🤓Resources
    • Deep RL Resources
    • Contributing
    • Motivation
    • Help
    • Contact
Powered by GitBook
On this page
  • PrioritizedReplay
  • Source Documentation
  • Example Memory Spec

Was this helpful?

  1. 🔧Development
  2. Memory

PrioritizedReplay

PreviousReplayNextOnPolicyReplay

Last updated 5 years ago

Was this helpful?

PrioritizedReplay

Code:

extends from Replay by calculating prioritization for sampling experiences based on errors in Q-values estimation.

Suitable for off-policy algorithms.

Source Documentation

Refer to the class documentation and example memory spec from the source:

Example Memory Spec

This specification creates a PrioritizedReplay (off-policy) memory with a maximum capacity of 10,000 elements, with a batch size of 32, and CER is disabled. The alpha and epsilon parameters are specific to PER in computing the errors.

{
    ...
    "agent": [{
      "memory": {
        "name": "PrioritizedReplay",
        "alpha": 0.6,
        "epsilon": 0.001,
        "batch_size": 32,
        "max_size": 10000,
        "use_cer": false
      }
    }],
    ...
}

For more concrete examples of memory spec specific to algorithms, refer to the existing .

slm_lab/agent/memory/prioritized.py
Prioritized Experience Replay (PER)
slm_lab/agent/memory/prioritized.py#L87-L104
spec files