CNN - SLM Lab

Convolutional Neural Network
Code: slm_lab/agent/net/conv.py​
These networks take a single state as input and produce one or more outputs. It consists of zero or more convolutional layers, followed by zero or more dense layers. Finally, there are one or more dense output layers. CNNs excel at image processing and processing inputs with 2D or 3D structure. This makes them well suited for environments with pixel level inputs or inputs with a spatial component.
For more information on each of the elements see this tutorial on CNNs or the PyTorch documentation​
Source Documentation
Refer to the class documentation and example net spec from the source: slm_lab/agent/net/conv.py#L11-L75​
Example Net Spec
This specification instantiates a typical CNN for deep RL with 3 convolutional layers (input channels are automatically inferred):
32 output channels, kernel size (8 x 8), stride of 4, padding of 0, and dilation of (1 x 1)
64 output channels, kernel size (4 x 4), stride of 2, padding of 0, and dilation of (1 x 1)
32 output channels, kernel size (3 x 3), stride of 1, padding of 0, and dilation of (1 x 1)
Then, the output of the last convolutional layer is flattened and connected to the fully-connected hidden layers (this can be empty):
a layer of 512 units, then to a layer of 256 units
Lastly, the final layer is connected to an output layer that is automatically constructed by inferring the the algorithm and the environment.
The network uses linear (ReLU) activations, no activation for the output layer, and no weight initialization function. The rest of the spec is annotated below.
1
{
2
    ...
3
    "agent": [{
4
      ...
5
      "net": {
6
        "type": "ConvNet",
7
        "shared": false,  // whether to shared networks for Actor-Critic
8
        "conv_hid_layers": [
9
            [32, 8, 4, 0, 1],
10
            [64, 4, 2, 0, 1],
11
            [32, 3, 1, 0, 1]
12
        ],
13
        "fc_hid_layers": [512, 256],
14
        "hid_layers_activation": "relu",
15
        "out_layer_activation": null,
16
        "init_fn": null,  // weight initialization
17
        "normalize": false,  // whether to divide input by 255.0
18
        "batch_norm": false,  // whether to add batchnorm layers
19
        "clip_grad_val": 1.0,  // clip gradient by norm
20
        "loss_spec": {
21
          "name": "SmoothL1Loss"  // default loss function used for regression
22
        },
23
        "optim_spec": {  // the optimizer and its arguments
24
          "name": "Adam",
25
          "lr": 0.02
26
        },
27
        ...
28
      }
29
    }],
30
    ...
31
}
Copied!
For more concrete examples of net spec specific to algorithms, refer to the existing spec files.

MLP

RNN

Last modified 6mo ago

Copy link

Contents

Convolutional Neural Network

Source Documentation

Example Net Spec