icon_title1279×352 19.1 KB
Greetings everyone! I am happy to announce that my RL library, Machin, designed for PyTorch, is close to its first public debut after several months of hard development!
Machin is designed with the elegant torch style in mind, while aiming to cover most of the functions provided by Ray 26. It is hosted at https://github.com/iffiX/machin 407
Currently, Machin is at this development stage:
alpha-1: laying down a framework for all code
alpha-2: finish testing all modules                       
alpha-3: clean and update old examples               
alpha-3.5: add architecture docs, tutorial docs, etc.
beta-1: public tests, collect response and perfect design    < Machin is at here
first official release!

Algorithms
Machin is able to support a variety of RL algorithms, including:
Single agent algorithms:

Deep Q-Network (DQN) 72
Double DQN 19
Dueling DQN 5
RAINBOW 11
Deep Deterministic policy Gradient (DDPG) 13
Twin Delayed DDPG (TD3) 8
Hystereric DDPG (Modified from Hys-DQN) 2
Advantage Actor-Critic (A2C) 12
Proximal Policy Optimization (PPO) 25
Soft Actor Critic (SAC) 22

Multi-agent algorithms:

Multi-agent DDPG (MADDPG) 38

Massively parallel algorithms:

Asynchronous A2C (A3C) 20
APEX-DQN, APEX-DDPG 10
IMPALA 6
Augmented random search (ARS, non-gradient) 1

Enhancements:

Prioritized Experience Replay (PER) 4
Generalized Advantage Estimation (GAE) 5
Recurrent networks in DQN, etc. 8

Algorithms to be supported:

Generative Adversarial Imitation Learning (GAIL) 5
Evolution Strategies
Linear Optimization Methods
QMIX (multi agent) 8
Model-based methods

Parallel
Machin is capable of:

Distributed & asynchronus training: provide implementations for parameter servers and gradient reduction servers.
Advance RPC: Role based launching, service registraction, resource sharing. constructed on the basis of torch RPC.
Process level Parallelization, based on enhanced Processes, Threads, Pools, Events.
Model level Parallelization, Splitting(Not yet implemented) and Assignning model shards to devices, by heuristic.

Utilities

Model checking toolset: check for nan values, abnormal gradients etc in your model, based on hooks.
Logging videos, images, produced during your training
Model serialization, auto loading and management toolset. No longer worry about unmatched devices!
Many more.