Comment on page
Experiences are stored in a circular buffer which grows until the memory has reached capacity. Once the memory is at capacity the oldest experiences are deleted to make space for the newest.
Batches of size
batch_sizeare sampled from the entire memory. Sampling is random uniform unless "PrioritizedReplay" memory is used.
Suitable for off-policy algorithms.
This specification creates a Replay memory with a maximum capacity of 10,000 elements, i.e. it can store experiences for 10,000 time steps. When
memory.sample()is called, it will return a batch of 32 elements. We also specify the use of CER (Combined Experience Replay), which guarantees the latest experience is included in the samples.