# CNN

## Convolutional Neural Network

Code: [slm\_lab/agent/net/conv.py](https://github.com/kengz/SLM-Lab/blob/master/slm_lab/agent/net/conv.py)

These networks take a single state as input and produce one or more outputs. It consists of zero or more convolutional layers, followed by zero or more dense layers. Finally, there are one or more dense output layers. CNNs excel at image processing and processing inputs with 2D or 3D structure. This makes them well suited for environments with pixel level inputs or inputs with a spatial component.

For more information on each of the elements see this [tutorial](https://github.com/lgraesser/CNN-Tutorial) on CNNs or the [PyTorch documentation](https://pytorch.org/docs/stable/nn.html#convolution-layers)

### Source Documentation

Refer to the class documentation and example net spec from the source: [slm\_lab/agent/net/conv.py#L11-L75](https://github.com/kengz/SLM-Lab/blob/master/slm_lab/agent/net/conv.py#L11-L75)

### **Example Net Spec**

This specification instantiates a typical CNN for deep RL with 3 convolutional layers (input channels are automatically inferred):

* 32 output channels, kernel size (8 x 8), stride of 4, padding of 0, and dilation of (1 x 1)
* 64 output channels, kernel size (4 x 4), stride of 2, padding of 0, and dilation of (1 x 1)
* 32 output channels, kernel size (3 x 3), stride of 1, padding of 0, and dilation of (1 x 1)

Then, the output of the last convolutional layer is flattened and connected to the fully-connected hidden layers (this can be empty):

* a layer of 512 units, then to a layer of 256 units

Lastly, the final layer is connected to an output layer that is automatically constructed by inferring the the algorithm and the environment.

The network uses linear (ReLU) activations, no activation for the output layer, and no weight initialization function. The rest of the spec is annotated below.

```javascript
{
    ...
    "agent": {
      ...
      "net": {
        "type": "ConvNet",
        "shared": true,  // whether to share networks for Actor-Critic
        "conv_hid_layers": [
            [32, 8, 4, 0, 1],
            [64, 4, 2, 0, 1],
            [32, 3, 1, 0, 1]
        ],
        "fc_hid_layers": [512],
        "hid_layers_activation": "relu",
        "out_layer_activation": null,
        "init_fn": null,  // weight initialization
        "batch_norm": false,  // whether to add batchnorm layers
        "clip_grad_val": 1.0,  // clip gradient by norm
        "loss_spec": {
          "name": "SmoothL1Loss"  // default loss function used for regression
        },
        "optim_spec": {  // the optimizer and its arguments
          "name": "AdamW",
          "lr": 0.00025
        },
        "gpu": "auto"
      }
    },
    ...
}
```

For more concrete examples of net spec specific to algorithms, refer to the existing [spec files](https://github.com/kengz/SLM-Lab/tree/master/slm_lab/spec).


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://slm-lab.gitbook.io/slm-lab/development/neural-networks/cnn.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.