Body: <p>I’ve been playing a game called Town of Salem lately. The goal of the game is for the town to find and lynch all evils in the town. To lynch someone, they must be voted up to the gallows first. This gave me an idea. If we analyze the voting patterns, we can determine who is evil, and who isn’t. Evils won’t vote up other evils, but will vote up town. Town will vote up evils, but are less likely to vote up town.</p>
  <p>I came up with a naive way to determine how evil or good someone is.
  Note: EN means times not voting evil, T means times voting town, TN means times not voting town, and E means times voting evil.
  The evil score will be calculated as 2<em>EN + T.
  The good score will be calculated as 2</em>E + TN.
  However, I chose these biases on a whim, and not scientifically. Also, I’d prefer one measure that says whether you should trust someone or not, rather than two measures that might conflict with each other.</p>
  <p>Thus, I have arrived at my problem. What’s the best way to create a classification AI that takes in EN, T, TN, and E for a player and outputs whether they are evil or not? Should I use a random forest? Should I use a regular old neural network with 4 input nodes and 2 output nodes? How many hidden layers should I have?</p>
  <p>By the way, I’ll be training my model on 30 games worth of data, which represents voting data for around 400 players, so my model will be using supervised learning.</p>
  <p>Edit: Here are some notes about voting I glossed over. There are two types of voting: voting to put someone on trial, and voting whether someone is guilty or innocent. Town will vote up players to the stand, then the defendant will give a defense speech. Then town votes whether the person should die or not. My AI only cares about the first type of voting because most evils will vote guilty on other evils to not seem suspicious. Thus, the first form of voting is most reliable for determining who is evil.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-08-24T05:07:24.733Z
Comments.Text: Does the F-score make sense? https://en.wikipedia.org/wiki/F-score
Comments.UserDisplayName: Jaume Oliver Lafont
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-08-24T07:27:32.907Z
Comments.Text: You may want to detail the voting-system.  For example, do all eligible-voters have to vote?  When/how do you find out if a vote was for an evil or non-evil?  Do evils always not-vote for other evils, or do you just mean that they prefer to avoid it?  How large do the populations tend to be?  What portion of them tend to be evils -- is it a minority or are they more at-parity?  Are votes one-at-a-time, where there's only one voting-process from start-to-finish, or do they overlap or have multiple votes simultaneously?
Comments.UserDisplayName: Nat
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-08-24T07:32:38.637Z
Comments.Text: Also, how would you want to handle ambiguity?  For example, do you want to always have a best-guess?  Or, if you're uncertain (like in edge-cases where the evidence isn't particularly compelling either way), are you more interested in identifying stuff like if someone's probably-evil, or would you prefer to know if they're probably-not-evil?
Comments.UserDisplayName: Nat
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-08-24T07:34:41.243Z
Comments.Text: (If you've played "_Among Us_", the crew'll often be more conservative about voting-off players early in the game.  However, later in the game, if the imposters are at near-parity with non-imposters, then the crew may become more aggressive, voting based on best-guesses, because failing to take action would be too risky.)
Comments.UserDisplayName: Nat
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-08-24T21:39:31.543Z
Comments.Text: @Nat Added an edit explaining the vote system
Comments.UserDisplayName: Some Guy
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-08-24T21:42:16.953Z
Comments.Text: @JaumeOliverLafont I do not understand how the F-Score applies. It looks like a way to determine the accuracy of models, but my question is what type of model to use. This will be helpful when I am training my model, though, so thanks
Comments.UserDisplayName: Some Guy
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-08-25T06:52:13.197Z
Comments.Text: @Nat 1. You don't have to vote someone up for a trial if you do not want to. 2. At the end of the game a rolelist is displayed saying who is evil and who isn't. Also, if people die during the game, we can see if they are evil. 3. Smart evils will vote up other evils to act good, but most evils prefer not to vote up other evils. 4. There are 6 evils and 9 town. 5. Votes are conducted one at a time.
Comments.UserDisplayName: Some Guy
Tags: <machine-learning><classification><game-ai>
Title: Find evils with AI (social deduction games)
UserDisplayName: Some Guy
ThreadText: # Find evils with AI (social deduction games)
  
  (Asked by: username_0 on 2022-08-23)
  
  I’ve been playing a game called Town of Salem lately. The goal of the game is for the town to find and lynch all evils in the town. To lynch someone, they must be voted up to the gallows first. This gave me an idea. If we analyze the voting patterns, we can determine who is evil, and who isn’t. Evils won’t vote up other evils, but will vote up town. Town will vote up evils, but are less likely to vote up town.
  
  I came up with a naive way to determine how evil or good someone is.
  Note: EN means times not voting evil, T means times voting town, TN means times not voting town, and E means times voting evil.
  The evil score will be calculated as 2*EN + T.
  The good score will be calculated as 2*E + TN.
  However, I chose these biases on a whim, and not scientifically. Also, I’d prefer one measure that says whether you should trust someone or not, rather than two measures that might conflict with each other.
  
  Thus, I have arrived at my problem. What’s the best way to create a classification AI that takes in EN, T, TN, and E for a player and outputs whether they are evil or not? Should I use a random forest? Should I use a regular old neural network with 4 input nodes and 2 output nodes? How many hidden layers should I have?
  
  By the way, I’ll be training my model on 30 games worth of data, which represents voting data for around 400 players, so my model will be using supervised learning.
  
  Edit: Here are username_0 notes about voting I glossed over. There are two types of voting: voting to put someone on trial, and voting whether someone is guilty or innocent. Town will vote up players to the stand, then the defendant will give a defense speech. Then town votes whether the person should die or not. My AI only cares about the first type of voting because most evils will vote guilty on other evils to not seem suspicious. Thus, the first form of voting is most reliable for determining who is evil.
  
  **Tags:** machine-learning, classification, game-ai
  
  > **Comments:**
  > **username_1**: Does the F-score make sense? https://en.wikipedia.org/wiki/F-score  
  > **username_2**: You may want to detail the voting-system.  For example, do all eligible-voters have to vote?  When/how do you find out if a vote was for an evil or non-evil?  Do evils always not-vote for other evils, or do you just mean that they prefer to avoid it?  How large do the populations tend to be?  What portion of them tend to be evils -- is it a minority or are they more at-parity?  Are votes one-at-a-time, where there's only one voting-process from start-to-finish, or do they overlap or have multiple votes simultaneously?  
  > **username_2**: Also, how would you want to handle ambiguity?  For example, do you want to always have a best-guess?  Or, if you're uncertain (like in edge-cases where the evidence isn't particularly compelling either way), are you more interested in identifying stuff like if someone's probably-evil, or would you prefer to know if they're probably-not-evil?  
  > **username_2**: (If you've played "_Among Us_", the crew'll often be more conservative about voting-off players early in the game.  However, later in the game, if the imposters are at near-parity with non-imposters, then the crew may become more aggressive, voting based on best-guesses, because failing to take action would be too risky.)  
  > **username_0**: @username_2 Added an edit explaining the vote system  
  > **username_0**: @username_1 I do not understand how the F-Score applies. It looks like a way to determine the accuracy of models, but my question is what type of model to use. This will be helpful when I am training my model, though, so thanks  
  > **username_0**: @username_2 1. You don't have to vote someone up for a trial if you do not want to. 2. At the end of the game a rolelist is displayed saying who is evil and who isn't. Also, if people die during the game, we can see if they are evil. 3. Smart evils will vote up other evils to act good, but most evils prefer not to vote up other evils. 4. There are 6 evils and 9 town. 5. Votes are conducted one at a time.  
  
  ---
  
  ## 0 answers
  
  