SLM Lab
Search…
Using SLM Lab In Your Project

SLM Lab As a Python Pip Module

This is an advanced use case tutorial.
The modular design of SLM Lab implies that its components can also be used in other projects. In fact, SLM Lab can be installed as a pip module and used like a typical Python package. This means you can just do a pip import slm_lab (orpython setup.py install without pip) and initialize an RL agent for use in your Python project.
This is especially crucial for those who want to use these algorithms in an industrial application. Often the app is part of a massive industrial system, and it is difficult or impossible to wrap that inside the lab. The agent must be made into an importable module to be used inside the app, either for training or for inferencing in deployment.
For demonstration, we have created a standalone script to show how to do this. The solution is very lightweight. The proper spec format will initialize the agent as usual, and as long as the proper agent APIs are called, all agent functionalities will work. Of course, to make use of the lab’s full potential such as distributed training and parameter search, you would still need to use SLM Lab.
The demo below uses a simplified form of the SLM Lab’s Session class. This shows that the main control loop and API methods are already generic.
1
# Installation:
2
# 1. Clone SLM-Lab
3
git clone https://github.com/kengz/SLM-Lab.git
4
cd SLM-Lab
5
​
6
# 2. Install SLM Lab as pip module
7
pip install -e .
Copied!
Let's see how we can implement a Session in an external project.
1
import os
2
# NOTE increase if needed. Pytorch thread overusage https://github.com/pytorch/pytorch/issues/975
3
os.environ['OMP_NUM_THREADS'] = '1'
4
from slm_lab.spec import spec_util
5
from slm_lab.lib import logger, util
6
from slm_lab.experiment import analysis
7
from slm_lab.env.openai import OpenAIEnv
8
from slm_lab.agent import Agent, Body
9
import torch
10
​
11
​
12
class Session:
13
'''A very simple Session that runs an RL loop'''
14
​
15
def __init__(self, spec):
16
self.spec = spec
17
self.env = OpenAIEnv(self.spec)
18
body = Body(self.env, self.spec)
19
self.agent = Agent(self.spec, body=body)
20
logger.info(f'Initialized session')
21
​
22
def run_rl(self):
23
clock = self.env.clock
24
state = self.env.reset()
25
done = False
26
while clock.get('frame') <= self.env.max_frame:
27
if done: # reset when episode is done
28
clock.tick('epi')
29
state = self.env.reset()
30
done = False
31
clock.tick('t')
32
with torch.no_grad():
33
action = self.agent.act(state)
34
next_state, reward, done, info = self.env.step(action)
35
self.agent.update(state, action, reward, next_state, done)
36
state = next_state
37
if clock.get('frame') % self.env.log_frequency == 0:
38
self.agent.body.ckpt(self.env, 'train')
39
self.agent.body.log_summary('train')
40
​
41
def close(self):
42
self.agent.close()
43
self.env.close()
44
logger.info('Session done and closed.')
45
​
46
def run(self):
47
self.run_rl()
48
# this will run SLM Lab's built-in analysis module and plot graphs
49
self.data = analysis.analyze_session(self.spec, self.agent.body.train_df, 'train')
50
self.close()
51
return self.data
52
​
53
​
54
# This uses SLM-Lab's existing spec. Alternatively, you can write one yourself too (see documentation for detail).
55
spec = spec_util.get(spec_file='slm_lab/spec/demo.json', spec_name='dqn_cartpole')
56
os.environ['lab_mode'] = 'train' # set to 'dev' for rendering
57
​
58
# update the tracking indices
59
spec_util.tick(spec, 'trial')
60
spec_util.tick(spec, 'session')
61
​
62
# initialize and run session
63
session = Session(spec)
64
session_metrics = session.run()
Copied!
Copy link