SLM Lab
Search…
SLM Lab
v4.2.3
SLM Lab
🖥Setup
Installation
Quick Start
🚀Using SLM Lab
Lab Command
Lab Organization
Train: REINFORCE CartPole
Resume and Enjoy: REINFORCE CartPole
Agent Spec: DDQN+PER on LunarLander
Env Spec: A2C on Pong
GPU Usage: PPO on Pong
Parallelizing Training: Async SAC on Humanoid
Experiment and Search Spec: PPO on Breakout
Run Benchmark: A2C on Atari Games
Meta Spec: High Level Specifications
Post-Hoc Analysis
TensorBoard: Visualizing Models and Actions
Using SLM Lab In Your Project
📈Analyzing Results
Data Locations
Graphs and Data
Performance Metrics
🥇Benchmark Results
Public Benchmark Data
Discrete Environment Benchmark
Continuous Environment Benchmark
Atari Environment Benchmark
RL GIFs
🔧Development
Modular Design
Algorithm
Memory
Net
MLP
CNN
RNN
Profiling SLM Lab
📖Publications and Talks
Book: Foundations of Deep Reinforcement Learning
Talks and Presentations
🤓Resources
Deep RL Resources
Contributing
Motivation
Help
Contact
Powered By GitBook
Net

​
🏗
 Net API

These networks are usable for all algorithms, and the lab takes care of the proper initialization with proper input/output sizing. One can swap out the network for any algorithm with just a spec change, e.g. make DQN into DRQN by substituting the net spec "type": "MLPNet" with "type": "RecurrentNet".
  • MLPNet
  • RecurrentNet
  • ConvNet
These networks are usable for Q-learning algorithms. For more details see this paper.
  • DuelingMLPNet
  • DuelingConvNet
Reinforcement learning (RL) algorithms typically involve approximating one or more unknown, complex, non linear functions. Deep neural networks make good candidates for these function approximators since they excel at approximating complex functions, particularly if the states are characterized by pixel level features.
Neural networks consists of many simple computational operations grouped into sequential layers. For an introduction to neural networks see this article. For further reading see Neural Networks and Deep Learning.
There are a variety of families of neural networks, which organize computation differently and are specialized to different types of tasks. A number of the core neural network families are provided to use out of the box.
  • ​Multi-layered Perceptron (MLP): Takes a single state as input. These networks are composed of a sequence of dense (fully connected) layers. General purpose, simplest network. Well suited for environments with a low dimensional state space.
  • ​Convolutional Neural Network (CNN): Takes a single state as input. Consists of one or more convolutional layers, followed by one or more dense layers. Excels at image processing. Well suited for environments with pixel level inputs or inputs with a spatial component.
  • ​Recurrent Neural Network (RNN): Takes a sequence of states as input. Consists of zero or more state processing layers. The output from these layers are passed to a recurrent layers. Specialized to processes sequences. Well suited to environments in which making a decision about how to act in state s would benefit from knowing what states came previously.
  • Pending: Convolutional Recurrent Neural Network (CRNN): Combines RNN and CNN. Takes a sequence of states as input.
The structure of the inputs and outputs is task (environment) and algorithm dependent in RL. This is handled automatically for users. Users need to specify the structure of the hidden network layers, the activation function, and the optimization strategy, through the following parameters.
Previous
OnPolicyBatchReplay
Next
MLP
Last modified 6mo ago
Copy link