> For the complete documentation index, see [llms.txt](https://slm-lab.gitbook.io/slm-lab/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://slm-lab.gitbook.io/slm-lab/master/development/modular-lab-components/class-inheritance-a2c-greater-than-ppo.md).

# Class Inheritance: A2C > PPO

## :herb: REINFORCE > A2C (Actor-Critic) > PPO

To showcase of the taxonomy-based/class inheritance implementation in SLM Lab, Proximal Policy Optimization (PPO) [(Schulman et al., 2017)](https://arxiv.org/abs/1707.06347) is a good example. When considered as a stand alone algorithm, PPO has a number of different components. However, it differs from the Actor-Critic algorithm only in how it computes the policy loss, runs the training loop, and by needing to maintain an additional actor network during training. Figure below shows how this similarity is reflected in the SLM Lab implementation of PPO:

![](/files/-Lu8o_3mjf7ZleB5dzmv)

The result is that the PPO class in SLM Lab has five overridden methods and contains only about [140 lines of code (excluding comments)](https://github.com/kengz/SLM-Lab/blob/master/slm_lab/agent/algorithm/ppo.py). Implementing it was straightforward once the [ActorCritic class](https://github.com/kengz/SLM-Lab/blob/master/slm_lab/agent/algorithm/actor_critic.py) was already implemented and thoroughly tested. More importantly, we can be sure that the performance difference between Actor-Critic and PPO observed in experiments using SLM Lab are due to something in the 140 lines of code that differentiate ActorCritic and PPO, and not to other implementation differences.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://slm-lab.gitbook.io/slm-lab/master/development/modular-lab-components/class-inheritance-a2c-greater-than-ppo.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.