Answers.Body: <p>The direct reason that your AI always moves the top-left piece first (assuming the computer pieces take up the bottom 2 rows) is the way your model interacts with the environment.</p>
  <p>Because you score every column separately, identical columns will always have identical scores. The way that you pick the best score means that if several columns have the same score, you will pick the left one, and if several rows have the same score, you will pick the top one. At the start of the game, since all the column scores are forced to be the same, that means it will always pick the first column. And since the AI pieces are in two rows, and those two rows are the same, and therefore must have the same scores, that means it will always pick the first row.</p>
  <p>(Note: I didn't actually run your code so my understanding could be slightly different from what actually happens, like if the AI is at the top of the screen or if the game starts with the player moving first. The same principles should still apply)</p>
  <hr />
  <p>Additionally, I don't think this is a good way to have the AI model interact with the environment. Imagine you are playing with your friends Billy and Sally. Every time you want to make a move, you print out two copies of the game board, cut one up into rows and give the rows to Billy, cut one up into columns and give the columns to Sally. Then you ask Billy to pick his favourite column and Sally to pick her favourite row and if there isn't a valid piece to move in that row and column, you slap them both.</p>
  <p>At best, they're just going to learn to try not to get slapped. Probably by liking the rows and columns with the most pieces in them, which gives them the greatest chance that there is a piece at the location the other one picks.</p>
  <p>And it's even worse because Billy and Sally are the same person and they don't know which cutouts are rows and which cutouts are columns.</p>
  <p>This is clearly not a good way to play.</p>
  <hr />
  <p>Your board game falls into the same category as Chess, Go and Tic-Tac-Toe - it has a game board with a finite (but large) number of states, a finite number of moves that the player can do according to the current state of the game board, you can win or lose or draw, and so on. There are not many moves or board states in Tic-Tac-Toe, more in Chess, and even more in Go.</p>
  <p>Board games of this type are traditionally played with any kind of <a href="https://en.wikipedia.org/wiki/Minimax" rel="nofollow noreferrer">minimax algorithm</a>, which looks at all the possible moves the computer could do, all the possible moves the player could do after that computer move, all the possible moves the computer could do next, and so on. Because there are way too many possible moves to check all of them (except in Tic-Tac-Toe), the computer has to <em>skip</em> most of the possibilities, and after looking a certain number of moves ahead it has to <em>estimate</em> how likely it is to win, instead of searching even farther ahead.</p>
  <p>Both of these are where a perceptron could come in. If you can train a perceptron to predict <em>how likely the computer is to win</em>, you can ignore computer moves that make it unlikely to win, player moves that make the computer likely to win, and you can use it as your estimate when you decide to stop searching deeper.</p>
  <p>The simplest version of this tree search algorithm just searches one move ahead and there is no reason to discard anything:</p>
  <pre><code>def get_next_move(board):
      best_move, best_score = None, -Infinity
      for move in all_valid_moves_for_computer(board):
          board.do(move) # only &quot;pretending&quot; to do the move, because we undo it afterwards
          move_score = call_perceptron(board) # if we did this move, how good is our situation?
          if move_score &gt; best_score:
              best_move, best_score = move, move_score
          board.undo(move)
      if best_move is None:
          # stalemate - no valid moves! game ends. do something about it here
      return best_move
  </code></pre>
  <p>A &quot;move&quot; means something you can do in the game, like moving a piece from a2 to a3. You could represent it in your program as <code>{&quot;from&quot;: (0,1), &quot;to&quot;: (0,2)}</code>. <code>all_valid_moves_for_computer</code> is a function that returns a list of all the moves the computer is allowed to make according to the rules of the game.</p>
  <hr />
  <p>Because your game has two different goals, you might want to train a different perceptron for each. Or, you can use the same one in reverse: if you only have a perceptron that tells you whether the computer will win as a defender, but the computer is playing as the attacker, then you can <em>pretend</em> the attacker is the computer (by turning the player pieces into computer pieces, computer pieces into player pieces, and turning the board upside down), check how the perceptron would like to win, and then do the <em>opposite</em> of what it says (do the lowest scored move).</p>
  <p>You might get away with using the same one for both, but I expect the gameplay is quite different as attacker and defender and the computer needs to be able to learn that.</p>
  <hr />
  <p>Whatever algorithm you use, don't forget that you have to train it to <em>win</em> the game, not just to make valid moves. Actually in the tree search algorithm (see above), you have normal code that figures out what the valid moves are. Normal computer programming is already pretty good at that. The perceptron's job is to see how good the computer's situation is if it does the move. You want it to have a high score when the computer is going to win and a low score when the computer is going to lose.</p>
  <p>You can do this by playing <em>and recording</em> a whole bunch of games against human friend or against yourself. Then you train the perceptron to predict whether the attacker will win or lose, just by looking at the board. Each time one player makes a move, that's a piece of training data, and at the end of the game you find out whether the labels for all those pieces of training data were 1 (more likely to win the game) or -1 (more likely to lose the game).</p>
  <hr />
  <p>If you are brave, you can also try a multi-layer perceptron. It will need more and better-quality training data.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-09-18T02:30:10.237Z
Answers.Text: I guess I had forgotten to close the question, but yeah, since you were inclined to help me, lemme give your helpfulness an upvote :D and accept the answer.
Answers.UserDisplayName: Spectre
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-09-18T02:32:35.123Z
Answers.Text: And yes, feature extraction worked better than anything and I have started recording moves. Also, I have somehow avoided infinite loops so it's running better than ever.. Thanks a lot for that kindness to help a learner! I'll certainly include your name in my team's synopsis, so please do let me know your name :).
Answers.UserDisplayName: Spectre
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-10-11T06:42:53.110Z
Answers.Text: I guess the chatroom was frozen, but never mind.. the question is still available, so I'll access you via this....
Answers.UserDisplayName: Spectre
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-10-11T06:43:54.587Z
Answers.Text: As of now, it's working like it's not me who built it... really superb. Also, I have started to code the other moves of the game into it, so I will let you know once it ends in some land of confusion
Answers.UserDisplayName: Spectre
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-10-11T08:21:19.300Z
Answers.Text: @Spectre since there are no private messages I won't include my name. You can perhaps link to my Stack Exchange profile page: https://stackexchange.com/users/37136/immibis
Answers.UserDisplayName: Criticize SE actions means ban
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-10-11T10:02:17.063Z
Answers.Text: @user253571 okay then, I'll use your pseudonym in my team's synopsis for your real name :D Thanks a lot for that great help again!
Answers.UserDisplayName: Spectre
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-10-11T11:08:48.897Z
Answers.Text: I meant the same... :)
Answers.UserDisplayName: Spectre
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-12-20T07:30:57.180Z
Answers.Text: here's my completed code (the link in the chatroom given below): https://chat.stackexchange.com/rooms/132477/a-cross-boundary-chatroom
Answers.UserDisplayName: Spectre
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-12-20T07:32:09.087Z
Answers.Text: I haven't been able to integrate AI into the game just because the cod had to be limited to 4 pages in the synopsis document as per what my teacher said. Also, the code was getting messy, hence I decided I look up alternatives from the rest of my teammates.
Answers.UserDisplayName: Spectre
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-09-17T13:13:10.150Z
Answers.LastActivityDate: 2021-09-17T13:30:12.657Z
Answers.LastEditDate: 2021-09-17T13:30:12.657Z
Answers.UserDisplayName: Criticize SE actions means ban
Body: <p>I was working a little bit on a school project my team and I decided to do for submission in the year-end. It's a small game which I call 'Quattro', and its rules are as follows:</p>
  <ol>
  <li>The game is played on an 8 x 8 square grid and each player (both the human and the computer) have sixteen pieces on their side (just the same layout as in chess, but here all pieces are identical for each player).</li>
  <li>Only vertical moves,(i.e., one can move only one square forward at a time and that too in the forward direction/along a column) as long as no other piece stands before the piece to be moved.</li>
  <li>However, one can cross over to a square present in the north-west or north-east (when you look around a 3 x 3 grid with the piece under consideration at the centre) if the north and west/east boundary of the piece when in the 3 x 3 grid are held by the enemy's pieces, as in the case of the move 'en passant' in chess. In the process, the enemy piece in the square just below the square the player has crossed over to is lost by the enemy.</li>
  <li>The players involved can either be an attacker or a defender (if one choses the former, the other takes the latter). The attacker wins if he/she successfully takes at least four of his/her pieces(hence the name Quattro) to the enemy's side (that is, to the last row counting from that player's side) while the defender wins if the attacker is prevented from doing so.</li>
  </ol>
  <p>You can request me to add screenshots in case the rules are very vague (even my teammates were confused ).</p>
  <p>Okay, so I'm doing this on Python 3.9.6 and I have somehow made the board layout and movement rules (except for rule 2 and rule 4, which are supposed to be added once the primary workings of the game is completed). I had somehow made the AI player (which is based on a single-layer perceptron), but I doubt if it is working right or not. The problem is that when I make a random move, the AI player starts always at the same column and moves pieces in some order I can't clearly remember(in the primary stage of creation, it all seemed to work fine, but as time progressed, I began to see indexing errors so I tried to adjust things somehow) and then it wanders off into an infinite loop. From a debug message I set up to observe the change in weights, I saw that at times one weight it growing while the other few would either be shrinking or remaining constant. As of now, I set up a variable to give the model a random target value (or may not be random, it seems) to train with and still the problem continues. I doubt if the input data is biased someway or the other. Here's how the input is taken:</p>
  <ol>
  <li>The model first checks through each column, and the input corresponding to each column will be a vector containing 0's and 1's with the 1's indicating the enemy's presence and 0 for the else case. The model thus generates a 'preference score' (equivalent to the activation function of the sum of the weighted inputs, as how it is in any other perceptron).</li>
  <li>The same is done for rows as well and the list of values in both cases are passed to a dictionary, from which the player choses the row and column index with the highest scores and moves the piece there.</li>
  </ol>
  <p>I also set up an <code>InvalidMove</code> exception so as to make sure that the machine doesn't play blankly.</p>
  <p>So here's the code:</p>
  <ul>
  <li>MarchOfTheFinalFour.py - the module containing the required exception and the board.</li>
  </ul>
  <pre><code># MarchOfThFinalFour.py
  
  from time import *
  
  # 'March of The Final Four' - a clone of chess
  
  player_piece = 'Φ' # player's piece
  computer_piece = 'τ' # computer's piece
  
  class InvalidMove(Exception):
      '''error when you/ the computer takes an invalid move'''
      def __init__(self, coords):
          self.stmt = &quot;invalid move from:&quot;
          self.coords = coords
  
  class PlayTable:
      def __init__(self, table_side):
          ''' generates the game board, empty and with no pieces '''
          self.length = table_side
          self.table = [[0]*table_side]*table_side
  
      def __repr__(self):
          '''prints the table'''
          print()
          table_str = ''
          num = 0
          for row in self.table:
              table_str += str(num) + &quot; \t&quot;
              for piece in row:
                  table_str += str(piece) + &quot;|&quot;
              table_str += &quot;\n&quot;
              num += 1
          return table_str + &quot;\t0 1 2 3 4 5 6 7&quot;
          
      def reset(self):
          ''' resets the board/ places pieces on it '''
          for row in [0, 1] :
              self.table[row] = [computer_piece]*self.length
  
          for row in [self.length-2, self.length - 1] :
              self.table[row] = [player_piece]*self.length
          
          for row in range(2, self.length-2):
              self.table[row] = [0]*self.length
      def move_piece(self, coord, turn = 'player'):
          '''moves the piece at coord '''
          if self.table[coord[1]%self.length][coord[0]%self.length] != 0 and self.table[(coord[1] + (1 if turn == 'computer' else -1))%self.length][coord[0]%self.length] == 0:
              temp = self.table[coord[1]][coord[0]]
              self.table[coord[1]][coord[0]] = 0
              direction = 1 if turn == 'computer' else -1
              self.table[coord[1]+direction][coord[0]] = temp
              print(f&quot;Moved {temp} from {(coord[0] if coord[0] &gt;= 0 else 8 + coord[0],coord[1] if coord[1] &gt;= 0 else 8 + coord[1])}&quot;) # msg
          elif self.table[coord[1]%self.length][coord[0%self.length]] == 0 or self.table[(coord[1] + (-1)**(1 if turn == 'player' else 1))%self.length][coord[0]%self.length] != 0:
              raise InvalidMove(coord)
          elif turn == 'player' and self.table[coord[1]%self.length][coord[0]%self.length] == computer_piece:
              raise InvalidMove(coord)
          elif turn == 'computer' and self.table[coord[1]%self.length][coord[0]%self.length] == player_piece:
              raise InvalidMove(coord)
          
          
  board = PlayTable(8)
  board.reset()
  print(board)
  
  </code></pre>
  <ul>
  <li>TestGameML.py - sample game, NPC, single-layer perceptron, etc. all lies here:</li>
  </ul>
  <pre><code>from math import *
  from random import *
  import MarchOfTheFinalFour as mff
  
  ######################
  
  ## Math functions for our use in here
      
  def multiply(list_a, list_b):
      '''matrix multiplication and addition'''
      list_res = [list_a[n] * list_b[n] for n in range(len(list_a))]
      return fsum(list_res)
  
  def sig(x):
      '''logistic sigmoid function'''
      return exp(x)/(1+ exp(x))
  
  ##############################
  
  ## Neighbourhood search
  
  def neighbourhood(coords, board_length):
      '''generates the 3 x 3 grid that forms the neighbourhod of the required square'''
      axial_neighbours =  [(coords[0] + 1, coords[1]),(coords[0] - 1, coords[1]),
                          (coords[0], coords[1] + 1), (coords[0], coords[1] - 1)] # neighbours along NEWS directins
      diagonal_neighbours = [(coords[0] + 1, coords[1]+1),(coords[0] - 1, coords[1] - 1),
                             (coords[0]-1, coords[1] + 1), (coords[0]+1, coords[1] - 1)] #diagonal neighbours
      neighbours = axial_neighbours + diagonal_neighbours # supposed neighbours
      ## purging those coordinates with negative values in them:
      for i in range(len(neighbours)):
          if (neighbours[i][0] &lt; 0 or neighbours[i][0] &gt; board_length - 1) or (neighbours[i][1] &lt; 0 or neighbours[i][1] &gt; board_length - 1):
              neighbours[i] = 0
      while 0 in neighbours:
          neighbours.remove(0)
      
      return neighbours
  
  ########################
  
  # The NPC's brain
  
  class NPC_Brain:
      '''brain of the NPC ;), actually a single-layer perceptron '''
      def __init__(self,board_size):
          ''' Initialiser'''
          self.inputs = board_size # no. of input nodes for the neural network
          self.weights = [random() for i in range(self.inputs)] # random weights for each game
          self.column_scores = [] # column scores (for each column) - the 'liking' of the computer to move a piece in a column as the output
                                  # of the neural network's processing
          self.row_scores = [] #same here
          self.inputs_template_columns = [] # a container to hold the inputs to the neural network
          self.inputs_template_rows = [] # same here but for rows
      def process(self, board, threshold):
          '''forward-feeding'''
          # we begin by setting the lists to zero so as to make the computer forget the past state of the board and to look for the current state
          self.inputs_template_columns = []
          self.inputs_template_rows = []
          self.column_scores = []
          self.row_scores = []
          self.row_scores = []
          for column in range(self.inputs):
              scores = [1 if row[column] == mff.computer_piece else 0 for row in board] # checking for enemies in each column
              self.inputs_template_columns.append(scores) 
              score = sig(multiply(scores, self.weights)/threshold) # using the logistic sigmoid function to generate a liking for columns :D
              self.column_scores.append(score) # each column score is appended
          for row in range(self.inputs):
              scores = [1 if board[row][i] == mff.player_piece else 0 for i in range(self.inputs)] # checking for enemies in each column
              self.inputs_template_rows.append(scores) 
              score = sig(multiply(scores, self.weights)/threshold) # using the logistic sigmoid function to generate a liking for columns :D
              self.row_scores.append(score) # each column score is appended
          return {'columns':self.column_scores, 'rows':self.row_scores}
      def back_prop(self, learning_rate, target = 1):
          '''Back-propagation, with error function as squared-error function (target - error)**2'''
          for j in range(len(self.inputs_template_columns)):
              for i in range(self.inputs):
                  '''overfitting can occur, but still let's try this'''
                  self.weights[i] +=  learning_rate * 2 * (self.column_scores[j] - target) * (self.column_scores[j]*(1-self.column_scores[j])) * self.inputs_template_columns[j][i] #backprop formula
          for k in range(len(self.inputs_template_rows)):
              for i in range(self.inputs):
                  '''overfitting can occur, but still let's try this'''
                  self.weights[i] += learning_rate * 2 * (self.row_scores[k] - target) * (self.row_scores[k]*(1-self.row_scores[k])) * self.inputs_template_rows[k][i] #backprop formula
                  
      
          
  
  class NPC:
      ''' non-playable character / computerized player class '''
      def __init__(self):
          self.mind = NPC_Brain(mff.board.length) # the model
          self.piece_lower = 0; self.piece_upper = 1 # initial row numbers of the computer's pieces
          self.row_expanse = 2
          
      def make_move(self):
          moved = False
          req_target = 0.5
          counter = 1
          while not moved:
              if counter % 50 == 0:
                  req_target += log(req_target**(counter%25))
                  print(&quot;New target set:&quot;, req_target)
              score_board = temp = self.mind.process(mff.board.table, 0.5) # feeding forward
              x_coord = score_board['columns'].index(max(score_board['columns'])) # choosing the column the compute likes the most
              y_coord = score_board['rows'].index(max(score_board['rows'])) % self.row_expanse # a random y coordinate is chosen
              try:
                  if y_coord &lt; mff.board.length - 1: 
                      if mff.board.table[int(y_coord) + 1][int(x_coord)] == 0 and (mff.board.table[int(y_coord)][int(x_coord)] not in  [0, mff.player_piece]):
                          mff.board.move_piece((int(x_coord), int(y_coord)), turn = 'computer')
                          self.piece_upper += 1 #increasing the upper limit of the y coordinate by 1
                          moved = True
                          req_target += 0.0001
                          self.row_expanse += 1
                          counter += 1
                      else:
                          raise mff.InvalidMove((x_coord,y_coord))
                      counter += 1 
              except mff.InvalidMove:
                  # trying to avoid the computer's confusion
                  self.mind.back_prop(1/pi, target = req_target) # making the computer learn from its decision
                  req_target -= 0.0001
                  counter += 1
                      
              
                  
              
                  
  
  
  npc = NPC() # creating the NPC
  
  ## Sample gamplay
  ## The following gameplay will be a bit smooth in the beginning but turns into a confusion later
  all_gone_good = True
  while True:
      all_gone_good = True 
      # infinite loop here till errors occur
      player_mv = eval(input(&quot;Enter your move:&quot;)) # waiting for the player's move
      try:
          mff.board.move_piece(player_mv)
      except mff.InvalidMove:
          print(&quot;Invalid move&quot;)
          all_gone_good = False
      # next we check if the player's move was valid
      if all_gone_good:
          print(mff.board)
          npc.make_move()
          print(mff.board)
          
  </code></pre>
  <p>I am sorry that  haven't been able to comment in certain regions of the code, in which case you can ask me for clarification.</p>
  <p>My main doubts are : is my data acquisition method biased? Is the training part also little bit wacky? Or is it that I programmed it all without knowing what I am doing? What's actually causing such an infinite loop?</p>
  <hr />
  <p><em><em>Edit:</em></em> : I have edited TestGameML.py and it's down here:</p>
  <pre><code>from math import *
  from random import *
  import MarchOfTheFinalFour as mff
  
  ######################
  ##Bug fixes required:
  
  ##1. The machine is making multiple moves unknowingly
  
  ######################
  
  ## Some variables for global use
  
  my_move = (0,0)
  
  ## Math functions for our use in here
      
  def multiply(list_a, list_b):
      '''matrix multiplication and addition'''
      list_res = [list_a[n] * list_b[n] for n in range(len(list_a))]
      return fsum(list_res)
  
  def sig(x):
      '''logistic sigmoid function'''
      return exp(x)/(1+ exp(x))
  
  ##############################
  
  ## Neighbourhood search
  
  def neighbourhood(coords, board_length):
      '''generates the 3 x 3 grid that forms the neighbourhod of the required square'''
      axial_neighbours =  [(coords[0] + 1, coords[1]),(coords[0] - 1, coords[1]),
                          (coords[0], coords[1] + 1), (coords[0], coords[1] - 1)] # neighbours along NEWS directins
      diagonal_neighbours = [(coords[0] + 1, coords[1]+1),(coords[0] - 1, coords[1] - 1),
                             (coords[0]-1, coords[1] + 1), (coords[0]+1, coords[1] - 1)] #diagonal neighbours
      neighbours = axial_neighbours + diagonal_neighbours # supposed neighbours
      ## purging those coordinates with negative values in them:
      for i in range(len(neighbours)):
          if (neighbours[i][0] &lt; 0 or neighbours[i][0] &gt; board_length - 1) or (neighbours[i][1] &lt; 0 or neighbours[i][1] &gt; board_length - 1):
              neighbours[i] = 0
      while 0 in neighbours:
          neighbours.remove(0)
      
      return neighbours
  
  ########################
  # The NPC's brain
  
  class NPC_Brain:
      '''brain of the NPC ;), actually a single-layer perceptron '''
      def __init__(self,board_size):
          ''' Initialiser'''
          self.inputs = board_size # no. of input nodes for the neural network
          #self.weights = [random() for i in range(self.inputs)] random weights for each game
          self.weights = [0.5]*self.inputs
          self.column_scores = [] # column scores (for each column) - the 'liking' of the computer to move a piece in a column as the output
                                  # of the neural network's processing
          self.row_scores = [] #same here
          self.inputs_template_columns = [] # a container to hold the inputs to the neural network
          self.inputs_template_rows = [] # same here but for rows
          
      def process(self, board, threshold):
          '''forward-feeding'''
          # we begin by setting the lists to zero so as to make the computer forget the past state of the board and to look for the current state
          self.inputs_template_columns = []
          self.inputs_template_rows = []
          self.column_scores = []
          self.row_scores = []
          for column in range(self.inputs):
              scores = [(1/8)**(row + 1 if row == my_move[1] else 1) if board[row][column] == mff.player_piece else -1/8 for row in range(self.inputs)] # checking for enemies in each column
              self.inputs_template_columns.append(scores) 
              score = sig(multiply(scores, self.weights)/threshold) # using the logistic sigmoid function to generate a liking for columns :D
              self.column_scores.append(score) # each column score is appended
          for row in range(self.inputs):
              scores = [(1/8)**(i + 1 if i == my_move[0] else 1) if board[row][i] == mff.player_piece else -1/8 for i in range(self.inputs)] # checking for enemies in each column
              self.inputs_template_rows.append(scores) 
              score = sig(multiply(scores, self.weights)/threshold) # using the logistic sigmoid function to generate a liking for columns :D
              self.row_scores.append(score) # each column score is appended
          return {'columns':self.column_scores, 'rows':self.row_scores}
      
      def back_prop(self, learning_rate, target = 1):
          '''Back-propagation, with error function as squared-error function (target - error)**2'''
          for j in range(len(self.inputs_template_columns)):
              for i in range(self.inputs):
                  '''overfitting can occur, but still let's try this'''
                  self.weights[i] +=  -learning_rate * 2 * (self.column_scores[j] - target) * ((self.column_scores[j]**2)*(1-self.column_scores[j])) * self.inputs_template_columns[j][i] #backprop formula
          for k in range(len(self.inputs_template_rows)):
              for i in range(self.inputs):
                  '''overfitting can occur, but still let's try this'''
                  self.weights[i] += -learning_rate * 2 * (self.row_scores[k] - target) * ((self.row_scores[k]**2)*(1-self.row_scores[k])) * self.inputs_template_rows[k][i] #backprop formula
                  
      
          
  
  class NPC:
      ''' non-playable character / computerized player class '''
      def __init__(self):
          self.mind = NPC_Brain(mff.board.length) # the model
          self.piece_lower = 0; self.piece_upper = 1 # initial row numbers of the computer's pieces
          self.row_expanse = 2
          
      def make_move(self):
          moved = False
          req_target = 0.5
          counter = 1
          print(&quot;Thinking...&quot;)
          while not moved:
              score_board = temp = self.mind.process(mff.board.table, 0.5) # feeding forward
              x_coord = score_board['columns'].index(min(score_board['columns'])) # choosing the column the compute likes the most
              y_coord = score_board['rows'].index(max(score_board['rows'])) % self.row_expanse # a random y coordinate is chosen
              try:
                  if y_coord &lt; mff.board.length - 1: 
                      if mff.board.table[int(y_coord) + 1][int(x_coord)] == 0 and (mff.board.table[int(y_coord)][int(x_coord)] not in  [0, mff.player_piece]):
                          mff.board.move_piece((int(x_coord), int(y_coord)), turn = 'computer')
                          self.piece_upper += 1 #increasing the upper limit of the y coordinate by 1
                          moved = True
                          self.row_expanse += 1
                          counter += 1
                      else:
                          raise mff.InvalidMove((x_coord,y_coord))
                      counter += 1 
              except mff.InvalidMove:
                  # trying to avoid the computer's confusion
                  self.mind.back_prop(0.5, target = req_target) # making the computer learn from its decision
                  counter += 1
                      
              
                  
              
                  
  
  
  npc = NPC() # creating the NPC
  
  ## Sample gamplay
  ## The following gameplay will be a bit smooth in the beginning but turns into a confusion later
  all_gone_good = True
  while True:
      all_gone_good = True 
      # infinite loop here till errors occur
      player_mv = eval(input(&quot;Enter your move:&quot;)) # waiting for the player's move
      try:
          mff.board.move_piece(player_mv)
      except mff.InvalidMove:
          print(&quot;Invalid move&quot;)
          all_gone_good = False
      # next we check if the player's move was valid
      if all_gone_good:
          my_move = player_mv
          print(mff.board)
          npc.make_move()
          print(mff.board)
  
  </code></pre>
  <p>Changelog:</p>
  <ul>
  <li>Change data distribution method in lines 71 and 76</li>
  <li>Asked NPC to choose the column with the least column score and max row score.</li>
  </ul>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-09-08T10:23:51.373Z
Comments.Text: If two columns or rows score the same then your make_move function chooses the first one. It's common to use a softmax layer to turn the scores into a probability distribution, then pick randomly according to the distribution. For more complex NNs you might want to look into libraries that do the backprop for you.
Comments.UserDisplayName: Criticize SE actions means ban
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-09-08T10:35:34.270Z
Comments.Text: @user253751 so basically the problem is that somehow my data distribution is confusing for the machine to handle?
Comments.UserDisplayName: Spectre
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-09-08T11:32:12.550Z
Comments.Text: @user253751 also I am not looking forward to use open-source modules unless very much required, as I don't wish to make the project complex enough to baffle my teammates... but what about cumulative scoring [that is, the score provided in the current iteration is added to the pervious scores and hence the scoreboard is updated]?
Comments.UserDisplayName: Spectre
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-09-08T11:37:06.567Z
Comments.Text: Or perhaps manipulating the score using powers or logarithms so that scores vary some way or the other?
Comments.UserDisplayName: Spectre
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-09-08T11:45:39.717Z
Comments.Text: Oh btw, the exponentiation method of scoring seems to do some sort of trick, but the computer is still in confusion :D
Comments.UserDisplayName: Spectre
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-09-08T15:28:19.523Z
Comments.Text: Did you realize that most AI models that play games don't learn in a single game, but rather, they've already been trained on thousands or millions of other games, and then the trained AI code is put into the game? Untrained neural networks are totally useless.
Comments.UserDisplayName: Criticize SE actions means ban
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-09-08T15:53:15.377Z
Comments.Text: @user253751 so what's basically happening is that the neural network used here is untrained and hence is generating foolish moves and at times ending up in a mess?
Comments.UserDisplayName: Spectre
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-09-08T15:55:03.927Z
Comments.Text: So what should I do? I am not sure how I can use the Monte Carlo approach for game AI models. It was suggested to me, but I wasn't sure of how it can be implemented in this game, especially when one can't easily judge the wise moves and unwise ones.
Comments.UserDisplayName: Spectre
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-09-08T15:57:18.127Z
Comments.Text: Also, I am currently in the beginning phase of the work so I haven't added any utilities to save configurations in files to continue searching for the optimal weights every time the code is run, with the weights in the previous run as the seed (I am not sure if that's against any principle in Machine Learning, and also being both an enthusiast and a beginner at the field, I'd like to be guided right). Perhaps adding such a system would do good?
Comments.UserDisplayName: Spectre
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-09-08T16:24:32.637Z
Comments.Text: I'm sorry for my late replies and slight impatience, I was attending a class ':)
Comments.UserDisplayName: Spectre
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-09-08T16:34:09.793Z
Comments.Text: You really do need to know a bunch of good moves to train the neural network so it can tell what's good and what's bad. I guess this is for a class project where you have to design some kind of AI for a game and they all play off against each other, and you chose to do it with a perceptron.
Comments.UserDisplayName: Criticize SE actions means ban
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-09-08T16:35:11.003Z
Comments.Text: I'm not sure the independent row and column approach is really very good. Especially it doesn't seem to make sense to use the same perceptron for scoring rows and columns. And I don't see how choosing the best scoring row and best scoring column *separately* makes much sense at all.
Comments.UserDisplayName: Criticize SE actions means ban
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-09-08T16:37:54.960Z
Comments.Text: If it does have to be done with a perceptron or you just want to use a perceptron for your version of the project, here is something I thought of - you could use a perceptron to score the "value" of a particular game board (which is probably much easier to learn) and then try all possible moves and see which one causes the perceptron to output the highest value after the move. Do that move. For training, you would have to play a whole bunch of games (against other humans probably) and aim to have it output 1 for all the boards that lead to winning, and 0 for all the boards that lead to losing.
Comments.UserDisplayName: Criticize SE actions means ban
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-09-08T16:40:00.600Z
Comments.Text: you could also try to generate training data by playing the AI against itself (but this needs a *good* AI!) or by playing some *other* AI (like alpha-beta pruning) against itself or against players. No matter how you slice it, you need a big set of recorded games as training data. At least 100, probably way more. If you aren't willing to find a way to get that data, you should probably give up now and try something different.
Comments.UserDisplayName: Criticize SE actions means ban
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-09-08T16:53:12.907Z
Comments.Text: Let us [continue this discussion in chat](https://chat.stackexchange.com/rooms/129402/discussion-between-spectre-and-user253751).
Comments.UserDisplayName: Spectre
Tags: <neural-networks><game-ai><single-layer-perceptron>
Title: Is my single layer perceptron getting biased input some way or the other?
UserDisplayName: Spectre
ThreadText: # Is my single layer perceptron getting biased input some way or the other?
  
  (Asked by: username_0 on 2021-09-08)
  
  I was working a little bit on a school project my team and I decided to do for submission in the year-end. It's a small game which I call 'Quattro', and its rules are as follows:
  
  The game is played on an 8 x 8 square grid and each player (both the human and the computer) have sixteen pieces on their side (just the same layout as in chess, but here all pieces are identical for each player).
  Only vertical moves,(i.e., one can move only one square forward at a time and that too in the forward direction/along a column) as long as no other piece stands before the piece to be moved.
  However, one can cross over to a square present in the north-west or north-east (when you look around a 3 x 3 grid with the piece under consideration at the centre) if the north and west/east boundary of the piece when in the 3 x 3 grid are held by the enemy's pieces, as in the case of the move 'en passant' in chess. In the process, the enemy piece in the square just below the square the player has crossed over to is lost by the enemy.
  The players involved can either be an attacker or a defender (if one choses the former, the other takes the latter). The attacker wins if he/she successfully takes at least four of his/her pieces(hence the name Quattro) to the enemy's side (that is, to the last row counting from that player's side) while the defender wins if the attacker is prevented from doing so.
  
  You can request me to add screenshots in case the rules are very vague (even my teammates were confused ).
  Okay, so I'm doing this on Python 3.9.6 and I have somehow made the board layout and movement rules (except for rule 2 and rule 4, which are supposed to be added once the primary workings of the game is completed). I had somehow made the AI player (which is based on a single-layer perceptron), but I doubt if it is working right or not. The problem is that when I make a random move, the AI player starts always at the same column and moves pieces in some order I can't clearly remember(in the primary stage of creation, it all seemed to work fine, but as time progressed, I began to see indexing errors so I tried to adjust things somehow) and then it wanders off into an infinite loop. From a debug message I set up to observe the change in weights, I saw that at times one weight it growing while the other few would either be shrinking or remaining constant. As of now, I set up a variable to give the model a random target value (or may not be random, it seems) to train with and still the problem continues. I doubt if the input data is biased someway or the other. Here's how the input is taken:
  
  The model first checks through each column, and the input corresponding to each column will be a vector containing 0's and 1's with the 1's indicating the enemy's presence and 0 for the else case. The model thus generates a 'preference score' (equivalent to the activation function of the sum of the weighted inputs, as how it is in any other perceptron).
  The same is done for rows as well and the list of values in both cases are passed to a dictionary, from which the player choses the row and column index with the highest scores and moves the piece there.
  
  I also set up an InvalidMove exception so as to make sure that the machine doesn't play blankly.
  So here's the code:
  
  MarchOfTheFinalFour.py - the module containing the required exception and the board.
  
  # MarchOfThFinalFour.py
  
  from time import *
  
  # 'March of The Final Four' - a clone of chess
  
  player_piece = 'Φ' # player's piece
  computer_piece = 'τ' # computer's piece
  
  class InvalidMove(Exception):
      '''error when you/ the computer takes an invalid move'''
      def __init__(self, coords):
          self.stmt = "invalid move from:"
          self.coords = coords
  
  class PlayTable:
      def __init__(self, table_side):
          ''' generates the game board, empty and with no pieces '''
          self.length = table_side
          self.table = [[0]*table_side]*table_side
  
      def __repr__(self):
          '''prints the table'''
          print()
          table_str = ''
          num = 0
          for row in self.table:
              table_str += str(num) + " \t"
              for piece in row:
                  table_str += str(piece) + "|"
              table_str += "\n"
              num += 1
          return table_str + "\t0 1 2 3 4 5 6 7"
          
      def reset(self):
          ''' resets the board/ places pieces on it '''
          for row in [0, 1] :
              self.table[row] = [computer_piece]*self.length
  
          for row in [self.length-2, self.length - 1] :
              self.table[row] = [player_piece]*self.length
          
          for row in range(2, self.length-2):
              self.table[row] = [0]*self.length
      def move_piece(self, coord, turn = 'player'):
          '''moves the piece at coord '''
          if self.table[coord[1]%self.length][coord[0]%self.length] != 0 and self.table[(coord[1] + (1 if turn == 'computer' else -1))%self.length][coord[0]%self.length] == 0:
              temp = self.table[coord[1]][coord[0]]
              self.table[coord[1]][coord[0]] = 0
              direction = 1 if turn == 'computer' else -1
              self.table[coord[1]+direction][coord[0]] = temp
              print(f"Moved {temp} from {(coord[0] if coord[0] >= 0 else 8 + coord[0],coord[1] if coord[1] >= 0 else 8 + coord[1])}") # msg
          elif self.table[coord[1]%self.length][coord[0%self.length]] == 0 or self.table[(coord[1] + (-1)**(1 if turn == 'player' else 1))%self.length][coord[0]%self.length] != 0:
              raise InvalidMove(coord)
          elif turn == 'player' and self.table[coord[1]%self.length][coord[0]%self.length] == computer_piece:
              raise InvalidMove(coord)
          elif turn == 'computer' and self.table[coord[1]%self.length][coord[0]%self.length] == player_piece:
              raise InvalidMove(coord)
          
          
  board = PlayTable(8)
  board.reset()
  print(board)
  
  
  
  TestGameML.py - sample game, NPC, single-layer perceptron, etc. all lies here:
  
  from math import *
  from random import *
  import MarchOfTheFinalFour as mff
  
  ######################
  
  ## Math functions for our use in here
      
  def multiply(list_a, list_b):
      '''matrix multiplication and addition'''
      list_res = [list_a[n] * list_b[n] for n in range(len(list_a))]
      return fsum(list_res)
  
  def sig(x):
      '''logistic sigmoid function'''
      return exp(x)/(1+ exp(x))
  
  ##############################
  
  ## Neighbourhood search
  
  def neighbourhood(coords, board_length):
      '''generates the 3 x 3 grid that forms the neighbourhod of the required square'''
      axial_neighbours =  [(coords[0] + 1, coords[1]),(coords[0] - 1, coords[1]),
                          (coords[0], coords[1] + 1), (coords[0], coords[1] - 1)] # neighbours along NEWS directins
      diagonal_neighbours = [(coords[0] + 1, coords[1]+1),(coords[0] - 1, coords[1] - 1),
                             (coords[0]-1, coords[1] + 1), (coords[0]+1, coords[1] - 1)] #diagonal neighbours
      neighbours = axial_neighbours + diagonal_neighbours # supposed neighbours
      ## purging those coordinates with negative values in them:
      for i in range(len(neighbours)):
          if (neighbours[i][0] < 0 or neighbours[i][0] > board_length - 1) or (neighbours[i][1] < 0 or neighbours[i][1] > board_length - 1):
              neighbours[i] = 0
      while 0 in neighbours:
          neighbours.remove(0)
      
      return neighbours
  
  ########################
  
  # The NPC's brain
  
  class NPC_Brain:
      '''brain of the NPC ;), actually a single-layer perceptron '''
      def __init__(self,board_size):
          ''' Initialiser'''
          self.inputs = board_size # no. of input nodes for the neural network
          self.weights = [random() for i in range(self.inputs)] # random weights for each game
          self.column_scores = [] # column scores (for each column) - the 'liking' of the computer to move a piece in a column as the output
                                  # of the neural network's processing
          self.row_scores = [] #same here
          self.inputs_template_columns = [] # a container to hold the inputs to the neural network
          self.inputs_template_rows = [] # same here but for rows
      def process(self, board, threshold):
          '''forward-feeding'''
          # we begin by setting the lists to zero so as to make the computer forget the past state of the board and to look for the current state
          self.inputs_template_columns = []
          self.inputs_template_rows = []
          self.column_scores = []
          self.row_scores = []
          self.row_scores = []
          for column in range(self.inputs):
              scores = [1 if row[column] == mff.computer_piece else 0 for row in board] # checking for enemies in each column
              self.inputs_template_columns.append(scores) 
              score = sig(multiply(scores, self.weights)/threshold) # using the logistic sigmoid function to generate a liking for columns :D
              self.column_scores.append(score) # each column score is appended
          for row in range(self.inputs):
              scores = [1 if board[row][i] == mff.player_piece else 0 for i in range(self.inputs)] # checking for enemies in each column
              self.inputs_template_rows.append(scores) 
              score = sig(multiply(scores, self.weights)/threshold) # using the logistic sigmoid function to generate a liking for columns :D
              self.row_scores.append(score) # each column score is appended
          return {'columns':self.column_scores, 'rows':self.row_scores}
      def back_prop(self, learning_rate, target = 1):
          '''Back-propagation, with error function as squared-error function (target - error)**2'''
          for j in range(len(self.inputs_template_columns)):
              for i in range(self.inputs):
                  '''overfitting can occur, but still let's try this'''
                  self.weights[i] +=  learning_rate * 2 * (self.column_scores[j] - target) * (self.column_scores[j]*(1-self.column_scores[j])) * self.inputs_template_columns[j][i] #backprop formula
          for k in range(len(self.inputs_template_rows)):
              for i in range(self.inputs):
                  '''overfitting can occur, but still let's try this'''
                  self.weights[i] += learning_rate * 2 * (self.row_scores[k] - target) * (self.row_scores[k]*(1-self.row_scores[k])) * self.inputs_template_rows[k][i] #backprop formula
                  
      
          
  
  class NPC:
      ''' non-playable character / computerized player class '''
      def __init__(self):
          self.mind = NPC_Brain(mff.board.length) # the model
          self.piece_lower = 0; self.piece_upper = 1 # initial row numbers of the computer's pieces
          self.row_expanse = 2
          
      def make_move(self):
          moved = False
          req_target = 0.5
          counter = 1
          while not moved:
              if counter % 50 == 0:
                  req_target += log(req_target**(counter%25))
                  print("New target set:", req_target)
              score_board = temp = self.mind.process(mff.board.table, 0.5) # feeding forward
              x_coord = score_board['columns'].index(max(score_board['columns'])) # choosing the column the compute likes the most
              y_coord = score_board['rows'].index(max(score_board['rows'])) % self.row_expanse # a random y coordinate is chosen
              try:
                  if y_coord < mff.board.length - 1: 
                      if mff.board.table[int(y_coord) + 1][int(x_coord)] == 0 and (mff.board.table[int(y_coord)][int(x_coord)] not in  [0, mff.player_piece]):
                          mff.board.move_piece((int(x_coord), int(y_coord)), turn = 'computer')
                          self.piece_upper += 1 #increasing the upper limit of the y coordinate by 1
                          moved = True
                          req_target += 0.0001
                          self.row_expanse += 1
                          counter += 1
                      else:
                          raise mff.InvalidMove((x_coord,y_coord))
                      counter += 1 
              except mff.InvalidMove:
                  # trying to avoid the computer's confusion
                  self.mind.back_prop(1/pi, target = req_target) # making the computer learn from its decision
                  req_target -= 0.0001
                  counter += 1
                      
              
                  
              
                  
  
  
  npc = NPC() # creating the NPC
  
  ## Sample gamplay
  ## The following gameplay will be a bit smooth in the beginning but turns into a confusion later
  all_gone_good = True
  while True:
      all_gone_good = True 
      # infinite loop here till errors occur
      player_mv = eval(input("Enter your move:")) # waiting for the player's move
      try:
          mff.board.move_piece(player_mv)
      except mff.InvalidMove:
          print("Invalid move")
          all_gone_good = False
      # next we check if the player's move was valid
      if all_gone_good:
          print(mff.board)
          npc.make_move()
          print(mff.board)
          
  
  I am sorry that  haven't been able to comment in certain regions of the code, in which case you can ask me for clarification.
  My main doubts are : is my data acquisition method biased? Is the training part also little bit wacky? Or is it that I programmed it all without knowing what I am doing? What's actually causing such an infinite loop?
  
  Edit: : I have edited TestGameML.py and it's down here:
  from math import *
  from random import *
  import MarchOfTheFinalFour as mff
  
  ######################
  ##Bug fixes required:
  
  ##1. The machine is making multiple moves unknowingly
  
  ######################
  
  ## Some variables for global use
  
  my_move = (0,0)
  
  ## Math functions for our use in here
      
  def multiply(list_a, list_b):
      '''matrix multiplication and addition'''
      list_res = [list_a[n] * list_b[n] for n in range(len(list_a))]
      return fsum(list_res)
  
  def sig(x):
      '''logistic sigmoid function'''
      return exp(x)/(1+ exp(x))
  
  ##############################
  
  ## Neighbourhood search
  
  def neighbourhood(coords, board_length):
      '''generates the 3 x 3 grid that forms the neighbourhod of the required square'''
      axial_neighbours =  [(coords[0] + 1, coords[1]),(coords[0] - 1, coords[1]),
                          (coords[0], coords[1] + 1), (coords[0], coords[1] - 1)] # neighbours along NEWS directins
      diagonal_neighbours = [(coords[0] + 1, coords[1]+1),(coords[0] - 1, coords[1] - 1),
                             (coords[0]-1, coords[1] + 1), (coords[0]+1, coords[1] - 1)] #diagonal neighbours
      neighbours = axial_neighbours + diagonal_neighbours # supposed neighbours
      ## purging those coordinates with negative values in them:
      for i in range(len(neighbours)):
          if (neighbours[i][0] < 0 or neighbours[i][0] > board_length - 1) or (neighbours[i][1] < 0 or neighbours[i][1] > board_length - 1):
              neighbours[i] = 0
      while 0 in neighbours:
          neighbours.remove(0)
      
      return neighbours
  
  ########################
  # The NPC's brain
  
  class NPC_Brain:
      '''brain of the NPC ;), actually a single-layer perceptron '''
      def __init__(self,board_size):
          ''' Initialiser'''
          self.inputs = board_size # no. of input nodes for the neural network
          #self.weights = [random() for i in range(self.inputs)] random weights for each game
          self.weights = [0.5]*self.inputs
          self.column_scores = [] # column scores (for each column) - the 'liking' of the computer to move a piece in a column as the output
                                  # of the neural network's processing
          self.row_scores = [] #same here
          self.inputs_template_columns = [] # a container to hold the inputs to the neural network
          self.inputs_template_rows = [] # same here but for rows
          
      def process(self, board, threshold):
          '''forward-feeding'''
          # we begin by setting the lists to zero so as to make the computer forget the past state of the board and to look for the current state
          self.inputs_template_columns = []
          self.inputs_template_rows = []
          self.column_scores = []
          self.row_scores = []
          for column in range(self.inputs):
              scores = [(1/8)**(row + 1 if row == my_move[1] else 1) if board[row][column] == mff.player_piece else -1/8 for row in range(self.inputs)] # checking for enemies in each column
              self.inputs_template_columns.append(scores) 
              score = sig(multiply(scores, self.weights)/threshold) # using the logistic sigmoid function to generate a liking for columns :D
              self.column_scores.append(score) # each column score is appended
          for row in range(self.inputs):
              scores = [(1/8)**(i + 1 if i == my_move[0] else 1) if board[row][i] == mff.player_piece else -1/8 for i in range(self.inputs)] # checking for enemies in each column
              self.inputs_template_rows.append(scores) 
              score = sig(multiply(scores, self.weights)/threshold) # using the logistic sigmoid function to generate a liking for columns :D
              self.row_scores.append(score) # each column score is appended
          return {'columns':self.column_scores, 'rows':self.row_scores}
      
      def back_prop(self, learning_rate, target = 1):
          '''Back-propagation, with error function as squared-error function (target - error)**2'''
          for j in range(len(self.inputs_template_columns)):
              for i in range(self.inputs):
                  '''overfitting can occur, but still let's try this'''
                  self.weights[i] +=  -learning_rate * 2 * (self.column_scores[j] - target) * ((self.column_scores[j]**2)*(1-self.column_scores[j])) * self.inputs_template_columns[j][i] #backprop formula
          for k in range(len(self.inputs_template_rows)):
              for i in range(self.inputs):
                  '''overfitting can occur, but still let's try this'''
                  self.weights[i] += -learning_rate * 2 * (self.row_scores[k] - target) * ((self.row_scores[k]**2)*(1-self.row_scores[k])) * self.inputs_template_rows[k][i] #backprop formula
                  
      
          
  
  class NPC:
      ''' non-playable character / computerized player class '''
      def __init__(self):
          self.mind = NPC_Brain(mff.board.length) # the model
          self.piece_lower = 0; self.piece_upper = 1 # initial row numbers of the computer's pieces
          self.row_expanse = 2
          
      def make_move(self):
          moved = False
          req_target = 0.5
          counter = 1
          print("Thinking...")
          while not moved:
              score_board = temp = self.mind.process(mff.board.table, 0.5) # feeding forward
              x_coord = score_board['columns'].index(min(score_board['columns'])) # choosing the column the compute likes the most
              y_coord = score_board['rows'].index(max(score_board['rows'])) % self.row_expanse # a random y coordinate is chosen
              try:
                  if y_coord < mff.board.length - 1: 
                      if mff.board.table[int(y_coord) + 1][int(x_coord)] == 0 and (mff.board.table[int(y_coord)][int(x_coord)] not in  [0, mff.player_piece]):
                          mff.board.move_piece((int(x_coord), int(y_coord)), turn = 'computer')
                          self.piece_upper += 1 #increasing the upper limit of the y coordinate by 1
                          moved = True
                          self.row_expanse += 1
                          counter += 1
                      else:
                          raise mff.InvalidMove((x_coord,y_coord))
                      counter += 1 
              except mff.InvalidMove:
                  # trying to avoid the computer's confusion
                  self.mind.back_prop(0.5, target = req_target) # making the computer learn from its decision
                  counter += 1
                      
              
                  
              
                  
  
  
  npc = NPC() # creating the NPC
  
  ## Sample gamplay
  ## The following gameplay will be a bit smooth in the beginning but turns into a confusion later
  all_gone_good = True
  while True:
      all_gone_good = True 
      # infinite loop here till errors occur
      player_mv = eval(input("Enter your move:")) # waiting for the player's move
      try:
          mff.board.move_piece(player_mv)
      except mff.InvalidMove:
          print("Invalid move")
          all_gone_good = False
      # next we check if the player's move was valid
      if all_gone_good:
          my_move = player_mv
          print(mff.board)
          npc.make_move()
          print(mff.board)
  
  
  Changelog:
  
  Change data distribution method in lines 71 and 76
  Asked NPC to choose the column with the least column score and max row score.
  
  **Tags:** neural-networks, game-ai, single-layer-perceptron
  > **Comments:**
  > **username_1**: If two columns or rows score the same then your make_move function chooses the first one. It's common to use a softmax layer to turn the scores into a probability distribution, then pick randomly according to the distribution. For more complex NNs you might want to look into libraries that do the backprop for you.  
  > **username_0**: @user253751 so basically the problem is that somehow my data distribution is confusing for the machine to handle?  
  > **username_0**: @user253751 also I am not looking forward to use open-source modules unless very much required, as I don't wish to make the project complex enough to baffle my teammates... but what about cumulative scoring [that is, the score provided in the current iteration is added to the pervious scores and hence the scoreboard is updated]?  
  > **username_0**: Or perhaps manipulating the score using powers or logarithms so that scores vary some way or the other?  
  > **username_0**: Oh btw, the exponentiation method of scoring seems to do some sort of trick, but the computer is still in confusion :D  
  > **username_1**: Did you realize that most AI models that play games don't learn in a single game, but rather, they've already been trained on thousands or millions of other games, and then the trained AI code is put into the game? Untrained neural networks are totally useless.  
  > **username_0**: @user253751 so what's basically happening is that the neural network used here is untrained and hence is generating foolish moves and at times ending up in a mess?  
  > **username_0**: So what should I do? I am not sure how I can use the Monte Carlo approach for game AI models. It was suggested to me, but I wasn't sure of how it can be implemented in this game, especially when one can't easily judge the wise moves and unwise ones.  
  > **username_0**: Also, I am currently in the beginning phase of the work so I haven't added any utilities to save configurations in files to continue searching for the optimal weights every time the code is run, with the weights in the previous run as the seed (I am not sure if that's against any principle in Machine Learning, and also being both an enthusiast and a beginner at the field, I'd like to be guided right). Perhaps adding such a system would do good?  
  > **username_0**: I'm sorry for my late replies and slight impatience, I was attending a class ':)  
  > **username_1**: You really do need to know a bunch of good moves to train the neural network so it can tell what's good and what's bad. I guess this is for a class project where you have to design some kind of AI for a game and they all play off against each other, and you chose to do it with a perceptron.  
  > **username_1**: I'm not sure the independent row and column approach is really very good. Especially it doesn't seem to make sense to use the same perceptron for scoring rows and columns. And I don't see how choosing the best scoring row and best scoring column *separately* makes much sense at all.  
  > **username_1**: If it does have to be done with a perceptron or you just want to use a perceptron for your version of the project, here is something I thought of - you could use a perceptron to score the "value" of a particular game board (which is probably much easier to learn) and then try all possible moves and see which one causes the perceptron to output the highest value after the move. Do that move. For training, you would have to play a whole bunch of games (against other humans probably) and aim to have it output 1 for all the boards that lead to winning, and 0 for all the boards that lead to losing.  
  > **username_1**: you could also try to generate training data by playing the AI against itself (but this needs a *good* AI!) or by playing some *other* AI (like alpha-beta pruning) against itself or against players. No matter how you slice it, you need a big set of recorded games as training data. At least 100, probably way more. If you aren't willing to find a way to get that data, you should probably give up now and try something different.  
  > **username_0**: Let us [continue this discussion in chat](https://chat.stackexchange.com/rooms/129402/discussion-between-spectre-and-user253751).  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2021-09-17. Score: 1 | Accepted answer
  
  The direct reason that your AI always moves the top-left piece first (assuming the computer pieces take up the bottom 2 rows) is the way your model interacts with the environment.
  Because you score every column separately, identical columns will always have identical scores. The way that you pick the best score means that if several columns have the same score, you will pick the left one, and if several rows have the same score, you will pick the top one. At the start of the game, since all the column scores are forced to be the same, that means it will always pick the first column. And since the AI pieces are in two rows, and those two rows are the same, and therefore must have the same scores, that means it will always pick the first row.
  (Note: I didn't actually run your code so my understanding could be slightly different from what actually happens, like if the AI is at the top of the screen or if the game starts with the player moving first. The same principles should still apply)
  
  Additionally, I don't think this is a good way to have the AI model interact with the environment. Imagine you are playing with your friends Billy and Sally. Every time you want to make a move, you print out two copies of the game board, cut one up into rows and give the rows to Billy, cut one up into columns and give the columns to Sally. Then you ask Billy to pick his favourite column and Sally to pick her favourite row and if there isn't a valid piece to move in that row and column, you slap them both.
  At best, they're just going to learn to try not to get slapped. Probably by liking the rows and columns with the most pieces in them, which gives them the greatest chance that there is a piece at the location the other one picks.
  And it's even worse because Billy and Sally are the same person and they don't know which cutouts are rows and which cutouts are columns.
  This is clearly not a good way to play.
  
  Your board game falls into the same category as Chess, Go and Tic-Tac-Toe - it has a game board with a finite (but large) number of states, a finite number of moves that the player can do according to the current state of the game board, you can win or lose or draw, and so on. There are not many moves or board states in Tic-Tac-Toe, more in Chess, and even more in Go.
  Board games of this type are traditionally played with any kind of minimax algorithm, which looks at all the possible moves the computer could do, all the possible moves the player could do after that computer move, all the possible moves the computer could do next, and so on. Because there are way too many possible moves to check all of them (except in Tic-Tac-Toe), the computer has to skip most of the possibilities, and after looking a certain number of moves ahead it has to estimate how likely it is to win, instead of searching even farther ahead.
  Both of these are where a perceptron could come in. If you can train a perceptron to predict how likely the computer is to win, you can ignore computer moves that make it unlikely to win, player moves that make the computer likely to win, and you can use it as your estimate when you decide to stop searching deeper.
  The simplest version of this tree search algorithm just searches one move ahead and there is no reason to discard anything:
  def get_next_move(board):
      best_move, best_score = None, -Infinity
      for move in all_valid_moves_for_computer(board):
          board.do(move) # only "pretending" to do the move, because we undo it afterwards
          move_score = call_perceptron(board) # if we did this move, how good is our situation?
          if move_score > best_score:
              best_move, best_score = move, move_score
          board.undo(move)
      if best_move is None:
          # stalemate - no valid moves! game ends. do something about it here
      return best_move
  
  A "move" means something you can do in the game, like moving a piece from a2 to a3. You could represent it in your program as {"from": (0,1), "to": (0,2)}. all_valid_moves_for_computer is a function that returns a list of all the moves the computer is allowed to make according to the rules of the game.
  
  Because your game has two different goals, you might want to train a different perceptron for each. Or, you can use the same one in reverse: if you only have a perceptron that tells you whether the computer will win as a defender, but the computer is playing as the attacker, then you can pretend the attacker is the computer (by turning the player pieces into computer pieces, computer pieces into player pieces, and turning the board upside down), check how the perceptron would like to win, and then do the opposite of what it says (do the lowest scored move).
  You might get away with using the same one for both, but I expect the gameplay is quite different as attacker and defender and the computer needs to be able to learn that.
  
  Whatever algorithm you use, don't forget that you have to train it to win the game, not just to make valid moves. Actually in the tree search algorithm (see above), you have normal code that figures out what the valid moves are. Normal computer programming is already pretty good at that. The perceptron's job is to see how good the computer's situation is if it does the move. You want it to have a high score when the computer is going to win and a low score when the computer is going to lose.
  You can do this by playing and recording a whole bunch of games against human friend or against yourself. Then you train the perceptron to predict whether the attacker will win or lose, just by looking at the board. Each time one player makes a move, that's a piece of training data, and at the end of the game you find out whether the labels for all those pieces of training data were 1 (more likely to win the game) or -1 (more likely to lose the game).
  
  If you are brave, you can also try a multi-layer perceptron. It will need more and better-quality training data.
  
  > **Comments:**
  > **username_0**: I guess I had forgotten to close the question, but yeah, since you were inclined to help me, lemme give your helpfulness an upvote :D and accept the answer.  
  > **username_0**: And yes, feature extraction worked better than anything and I have started recording moves. Also, I have somehow avoided infinite loops so it's running better than ever.. Thanks a lot for that kindness to help a learner! I'll certainly include your name in my team's synopsis, so please do let me know your name :).  
  > **username_0**: I guess the chatroom was frozen, but never mind.. the question is still available, so I'll access you via this....  
  > **username_0**: As of now, it's working like it's not me who built it... really superb. Also, I have started to code the other moves of the game into it, so I will let you know once it ends in some land of confusion  
  > **username_1**: @Spectre since there are no private messages I won't include my name. You can perhaps link to my Stack Exchange profile page: https://stackexchange.com/users/37136/immibis  
  > **username_0**: @user253571 okay then, I'll use your pseudonym in my team's synopsis for your real name :D Thanks a lot for that great help again!  
  > **username_0**: I meant the same... :)  
  > **username_0**: here's my completed code (the link in the chatroom given below): https://chat.stackexchange.com/rooms/132477/a-cross-boundary-chatroom  
  > **username_0**: I haven't been able to integrate AI into the game just because the cod had to be limited to 4 pages in the synopsis document as per what my teacher said. Also, the code was getting messy, hence I decided I look up alternatives from the rest of my teammates.  
  
  ---
  
  