# Modular Design

## :package: **Modular Design in SLM Lab**

Modularity is the central design choice in SLM Lab, as depicted in the figure below. Reinforcement learning algorithms in SLM Lab are built around three base classes:

* **Algorithm**: Handles interaction with the environment, implements an action policy, computes the algorithm-specific loss functions, and runs the training step.
* **Memory**: Provides the data storage and retrieval necessary for training.
* **Net**: Implements the deep networks that serve as the function approximators for an Algorithm.

The deep learning components in SLM Lab are implemented using PyTorch. The **Memory** and **Net** classes abstract data storage, data retrieval, and network training details, simplifying algorithm implementations. Furthermore, many Algorithm classes are natural extensions of each other.

![](https://2605794550-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LterKThgiKKTvsbx4WI%2F-Lu6MbpSMpj39Fcq7Enq%2F-Lu6ctUWhqWNK0TglURN%2Fslm_lab_class_diagram.png?alt=media\&token=74d0c72e-939e-43a2-97c6-0f57b6ce3b9b)

Modular code is critical for deep RL research because many RL algorithms are extensions of other RL algorithms. If two RL algorithms differ in only a small way, but a researcher compares their performance by running a standalone implementation of each algorithm, they cannot know whether differences in algorithm performance are due to meaningful differences between the algorithms or merely due to quirks in the two implementations. [Henderson et al. (2017)](https://arxiv.org/abs/1709.06560) showcase this, demonstrating significant performance differences between different implementations of the same algorithm.

Modular code is also important for research progress. It makes it as simple as possible for a researcher to implement — and reliably evaluate — new RL algorithms. And for the student of RL, modular code is easier to read and learn from due to its brevity and organization into digestible components.


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://slm-lab.gitbook.io/slm-lab/v4.2.0/development/modular-lab-components.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.