TensorBoard is built-in to SLM Lab. It will record all of the metric variables already logged to the terminal output, as well as the PyTorch model graph, model parameter histogram, and action histograms. All of these are done automatically during checkpointing using a writer in
agent.body. This allows for richer diagnosis of the network and policy, e.g. by seeing if the distributions shift over the course of learning.
Tensorboard event files are saved to the
log/ folder in the output data. During/after a run, you can launch TensorBoard for diagnosis:
Then, go to
localhost:6006 on your browser, and you should see the TensorBoard page:
The histogram tab is useful for revealing the distributions of the actions. In the example above, BipedalWalker has 4 continuous actions, hence there are 4 groups for plots for visualizing the value distributions of these 4 actions across different trials and sessions. Likewise, all the model parameters of an agent is also recorded as value distributions of the parameters of their layers.
In the histograms, the vertical axis (coming out from the page) is the number of frames during checkpoints. As an agent learns over time, we should see the distributions changing in shape and shifting locations.