Comment on page
Replay
Experiences are stored in a circular buffer which grows until the memory has reached capacity. Once the memory is at capacity the oldest experiences are deleted to make space for the newest.
Batches of size
batch_size are sampled from the entire memory. Sampling is random uniform unless "PrioritizedReplay" memory is used.Suitable for off-policy algorithms.
Refer to the class documentation and example memory spec from the source: slm_lab/agent/memory/replay.py#L43-L67
This specification creates a Replay memory with a maximum capacity of 10,000 elements, i.e. it can store experiences for 10,000 time steps. When
memory.sample() is called, it will return a batch of 32 elements. We also specify the use of CER (Combined Experience Replay), which guarantees the latest experience is included in the samples.{
...
"agent": [{
"memory": {
"name": "Replay",
"batch_size": 32,
"max_size": 10000,
"use_cer": true
}
}],
...
}
Last modified 2yr ago