Answers.Body: <p>Disclaimer: I have only experience with the first three frameworks.</p>
  <p>Among those three, I would suggest RLlib for the multi-agent situation:</p>
  <ol>
  <li>Distributed training based on Ray library</li>
  <li>Out-of-box multiagent solution</li>
  <li>Great documentation. You may want to check an example how to train  multiple agents: <a href="https://applied-rl-course.netlify.app/en/module2" rel="nofollow noreferrer">https://applied-rl-course.netlify.app/en/module2</a></li>
  </ol>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-08-19T15:19:20.753Z
Answers.LastActivityDate: 2023-08-19T15:31:26.297Z
Answers.LastEditDate: 2023-08-19T15:31:26.297Z
Answers.UserDisplayName: penkovsky
Body: <p>I have built a custom discrete event simulator with multiple agents and want to optimize the system using RL frameworks that support multi-agent configurations. I will use custom policies. Which framework should I use? I've looked into SB3, CleanRL, Tianshou, SKRL, RLlib, Acme, and MARLlib, and here's what I found:</p>
  <ol>
  <li><p>SB3 and CleanRL don't offer direct support for multi-agent systems.</p>
  </li>
  <li><p>RLlib is very functional but has a steep learning curve and hard to customize and some say the documentation is poor.</p>
  </li>
  <li><p>Tianshou seems good, but its community is small.</p>
  </li>
  <li><p>Acme doesn't use the PyTorch backend, which I prefer.</p>
  </li>
  <li><p>I haven't delved deeply into SKRL or MARLlib, but they appear promising.</p>
  </li>
  </ol>
  <p>I prioritize ease of use and documentation. What framework do you suggest? And why?</p>
  <p>I’d appreciate any helpful starting advice/resource to approach my problem as well.</p>
  
Tags: <reinforcement-learning><deep-rl><gym><multi-agent-rl>
Title: RL framework to optimize my custom multi-agent simulator
UserDisplayName: RookieScientist
ThreadText: # RL framework to optimize my custom multi-agent simulator
  
  (Asked by: username_0 on 2023-08-19)
  
  I have built a custom discrete event simulator with multiple agents and want to optimize the system using RL frameworks that support multi-agent configurations. I will use custom policies. Which framework should I use? I've looked into SB3, CleanRL, Tianshou, SKRL, RLlib, Acme, and MARLlib, and here's what I found:
  
  1. SB3 and CleanRL don't offer direct support for multi-agent systems.
  2. RLlib is very functional but has a steep learning curve and hard to customize and some say the documentation is poor.
  3. Tianshou seems good, but its community is small.
  4. Acme doesn't use the PyTorch backend, which I prefer.
  5. I haven't delved deeply into SKRL or MARLlib, but they appear promising.
  
  I prioritize ease of use and documentation. What framework do you suggest? And why?
  
  I’d appreciate any helpful starting advice/resource to approach my problem as well.
  
  **Tags:** reinforcement-learning, deep-rl, gym, multi-agent-rl
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2023-08-19. Score: 1
  
  Disclaimer: I have only experience with the first three frameworks.
  
  Among those three, I would suggest RLlib for the multi-agent situation:
  
  1. Distributed training based on Ray library
  2. Out-of-box multiagent solution
  3. Great documentation. You may want to check an example how to train multiple agents: <https://applied-rl-course.netlify.app/en/module2>
  
  
  ---
  
  