Memory is a class for data storage and access consistent with the RL agent API, i.e. implementing
sample methods. The underlying data format is numpy, which can efficiently be put into PyTorch tensors using shared memory via
torch.from_numpy. There are two types of memory in RL:
For off-policy algorithms
For on-policy algorithms:
Reinforcement Learning (RL) agents learn by acting in environments. Each time an agent acts, it stores
<s, a, s', r>, the state, action taken, next state, and reward received, as an experience in memory.
RL algorithms vary in the way they make use of past experiences, for example by sampling batches from the last N experiences the agent has had or by using all the experiences gathered since the agent was last trained.