Body: <p>In general, I see a lot of tutorials and examples of the use of Monte Carlo tree search in which only one agent is being considered (actions are not interleaved between different agents, as in the case of minimax, because only one agent is being considered), and the rollout results in win (+1) or lose (-1) or draw (0). However, it is not clear to me how to map this to a real-world scenario. When we have only one agent, if it wins at the end of the simulation, in my opinion, it is not necessary to simulate other scenarios. It is just a matter of following that course of action that resulted in a win.</p>
  <p>But, of course, there are examples of applications of MCTS in zero-sum two-player games. In these cases, the actions are interleaved, and each level of the tree represents information from the point of view of one of the agents, and the rollout results in win (+1) or lose (-1) or draw (0). These examples are clear to me.</p>
  <p>But the example with a single agent does not make any sense for me. Are there single-agent games in which we can use MCTS? Or are there other applications in which we can use MCTS for single-agent settings?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-07-23T20:24:11.387Z
Comments.Text: _if it wins at the end of the simulation_ MCTS very rarely gets to the bottom of the three seach in real world scenarios
Comments.UserDisplayName: Alberto
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-07-23T20:24:53.317Z
Comments.Text: and you can use MCTS in any problem where you have a model for the state transition
Comments.UserDisplayName: Alberto
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-07-24T14:24:51.440Z
Comments.Text: @Alberto 
  Yes, I read that we can use it in any problem where we have a model for the state transition. However, if, at the end of the simulation, the agent solves the problem (in a monoagent setting), it seems that it is not necessary to perform more simulations. Because, in this case, we have already found a solution. In this way, MCTS seems to be very similar to Depth First Search.
  
  Why should we continue performing simulations when the agent finds a solution for the problem?
Comments.UserDisplayName: Zaratruta
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-07-24T14:27:29.383Z
Comments.Text: @Alberto 
  You said that MCTS very rarely gets to the bottom of the three searches in real-world scenarios. This is not clear to me. Because, in most of the texts and tutorials I have read, They say that MCTS does not require heuristics, because it simulates the scenario until the end. Can you explain better what you said?
Comments.UserDisplayName: Zaratruta
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-07-24T16:25:00.123Z
Comments.Text: Simulate the game of life to the end and let's see how it goes...
Comments.UserDisplayName: Alberto
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-07-25T15:06:34.807Z
Comments.Text: @Alberto
  Game of life does not have an end state. I'm not referring to this kind of setting.
  Your comments are not helping to clarify the issues. Can you explain better? 
  My main concern is related to my first reply.
Comments.UserDisplayName: Zaratruta
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-07-26T09:21:08.800Z
Comments.Text: (1) in real-world scenario, MCTS might NEVER be able to do an exhaustive search, for example, in continuous action space (2) you continue because you have no idea what is the definition of what's a "good solution"
Comments.UserDisplayName: Alberto
Tags: <search><monte-carlo-tree-search>
Title: Example of use of monte carlo tree search in a monoagent task
UserDisplayName: Zaratruta
ThreadText: # Example of use of monte carlo tree search in a monoagent task
  
  (Asked by: username_0 on 2024-07-23)
  
  In general, I see a lot of tutorials and examples of the use of Monte Carlo tree search in which only one agent is being considered (actions are not interleaved between different agents, as in the case of minimax, because only one agent is being considered), and the rollout results in win (+1) or lose (-1) or draw (0). However, it is not clear to me how to map this to a real-world scenario. When we have only one agent, if it wins at the end of the simulation, in my opinion, it is not necessary to simulate other scenarios. It is just a matter of following that course of action that resulted in a win.
  
  But, of course, there are examples of applications of MCTS in zero-sum two-player games. In these cases, the actions are interleaved, and each level of the tree represents information from the point of view of one of the agents, and the rollout results in win (+1) or lose (-1) or draw (0). These examples are clear to me.
  
  But the example with a single agent does not make any sense for me. Are there single-agent games in which we can use MCTS? Or are there other applications in which we can use MCTS for single-agent settings?
  
  **Tags:** search, monte-carlo-tree-search
  > **Comments:**
  > **username_1**: _if it wins at the end of the simulation_ MCTS very rarely gets to the bottom of the three seach in real world scenarios  
  > **username_1**: and you can use MCTS in any problem where you have a model for the state transition  
  > **username_0**: @Alberto  Yes, I read that we can use it in any problem where we have a model for the state transition. However, if, at the end of the simulation, the agent solves the problem (in a monoagent setting), it seems that it is not necessary to perform more simulations. Because, in this case, we have already found a solution. In this way, MCTS seems to be very similar to Depth First Search.  Why should we continue performing simulations when the agent finds a solution for the problem?  
  > **username_0**: @Alberto  You said that MCTS very rarely gets to the bottom of the three searches in real-world scenarios. This is not clear to me. Because, in most of the texts and tutorials I have read, They say that MCTS does not require heuristics, because it simulates the scenario until the end. Can you explain better what you said?  
  > **username_1**: Simulate the game of life to the end and let's see how it goes...  
  > **username_0**: @Alberto Game of life does not have an end state. I'm not referring to this kind of setting. Your comments are not helping to clarify the issues. Can you explain better?  My main concern is related to my first reply.  
  > **username_1**: (1) in real-world scenario, MCTS might NEVER be able to do an exhaustive search, for example, in continuous action space (2) you continue because you have no idea what is the definition of what's a "good solution"  
  
  ---
  
  ## 0 answers
  
  