Answers.Body: <blockquote>
  <p>I wonder why these features are necessary, because I think a constant plane contains no information and it makes the the network larger and consequently harder to train.</p>
  </blockquote>
  <p>In many implementations of convolutional layers, the filters do not neatly remain inside the features plane when &quot;sliding&quot; along it, but (conceptually) also partially go &quot;outside&quot; the plane (where always at least one &quot;cell&quot; of the filter will still be inside the plane). Intuitively, in the case of a <span class="math-container">$3\times3$</span> filter for example, you can imagine that we pad the input features with an extra border of size <span class="math-container">$1$</span>, and this padding around the &quot;real&quot; input planes is filled with <span class="math-container">$0$</span>s.</p>
  <p>If it's possible for all input features to also have values of <span class="math-container">$0$</span>, it may in some situations be difficult or impossible for the neural network to distinguish the &quot;real&quot; <span class="math-container">$0$</span> inputs from the <span class="math-container">$0$</span> entries in the padding around the board, i.e. it may struggle to know where the game board is and where the game board ends. Having a constant input plane that's always filled with <span class="math-container">$1$</span>s can help in this respect, because that plane can always reliably be used to distinguish &quot;real&quot; cells that actually exist on the game board from positions that are just outside the game board.</p>
  <p>As for the plane filled with <span class="math-container">$0$</span>s... I have no idea why that would ever be useful. Maybe it was useful due to some peculiar implementation detail. In <a href="https://www.reddit.com/r/MachineLearning/comments/553jez/has_deepmind_released_their_alphago_network/d878p8u/" rel="nofollow noreferrer">this thread</a>, some people hypothesise that on specific hardware it might make some computations slightly more efficient because of the layout of the data in memory -- it causes the number of channels to be divisible by <span class="math-container">$8$</span>, which will.. maybe help? I really don't know too much about this, but I do know that on a smaller scale, <a href="https://software.intel.com/en-us/articles/coding-for-performance-data-alignment-and-structures" rel="nofollow noreferrer">sometimes adding unused data in classes can indeed increase performance due to better layout in memory</a>. I suppose it's also possible that it was accidentally added by mistake, or &quot;just in case&quot; and that it doesn't really have much of a purpose. The amount of hardware that the AlphaGo team had available to them is quite insane anyway, one channel more or less probably wasn't too big of a concern for them.</p>
  <hr />
  <blockquote>
  <p>What's more, I don't understand the sharp sign here. Does it mean &quot;the number&quot;? But one number is enough to represent &quot;the number of turns since a move was played&quot;, why eight?</p>
  </blockquote>
  <p>This is explained in the &quot;<strong>Features for policy/value network</strong>&quot; paragraph in the paper. Quote from the paper:</p>
  <blockquote>
  <p>&quot;Each integer feature value is split into multiple <span class="math-container">$19 \times 19$</span>&quot; planes of binary values (one-hot encoding). For example, separate binary feature planes ares used to represent whether an intersection has <span class="math-container">$1$</span> liberty, <span class="math-container">$2$</span> liberties, <span class="math-container">$\dots$</span>, <span class="math-container">$\geq 8$</span> liberties.</p>
  </blockquote>
  <p>All feature planes used were strictly binary, no feature planes were used that had integer values <span class="math-container">$&gt; 1$</span>. This is quite common when possible, because neural networks tend to have a much easier time learning with binary features than with integer- or real-valued features.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-03-01T11:16:33.270Z
Answers.LastActivityDate: 2021-01-05T20:30:43.180Z
Answers.LastEditDate: 2021-01-05T20:30:43.180Z
Answers.UserDisplayName: Dennis Soemers
Body: <p>In the paper <em>Mastering the game of Go with deep neural networks and tree search</em>, the input features of the networks of AlphaGo contains a plane of constant ones and a plane of constant zeros, as following.</p>
  
  <pre><code>Feature       #of planes Description 
  Stone colour  3          Player stone/opponent stone/empty 
  Ones          1          A constant plane ﬁlled with 1 
  Turns since   8          How many turns since a move was played 
  Liberties     8          Number of liberties (empty adjacent points) 
  Capture size  8          How many opponent stones would be captured 
  Self-atari size 8        How many of own stones would be captured 
  Liberties after move 8   Number of liberties after this move is played 
  Ladder capture 1         Whether a move at this point is a successful ladder capture 
  Ladder escape 1          Whether a move at this point is a successful ladder escape 
  Sensibleness  1          Whether a move is legal and does not ﬁll its own eyes 
  Zeros         1          A constant plane ﬁlled with 0 
  Player color  1          Whether current player is black
  </code></pre>
  
  <p>I wonder why these features are necessary, because I think a constant plane contains no information and it makes the the network larger and consequently harder to train.</p>
  
  <p>What's more, I don't understand the sharp sign here. Does it mean "the number"? But one number is enough to represent "the number of turns since a move was played", why eight?</p>
  
  <p>Thank you very much.</p>
  
Tags: <machine-learning><alphago>
Title: Why is a constant plane of ones added into the input features of AlphaGo?
UserDisplayName: Yangcy
ThreadText: # Why is a constant plane of ones added into the input features of AlphaGo?
  
  (Asked by: username_0 on 2019-03-05)
  
  In the paper Mastering the game of Go with deep neural networks and tree search, the input features of the networks of AlphaGo contains a plane of constant ones and a plane of constant zeros, as following.
  Feature       #of planes Description 
  Stone colour  3          Player stone/opponent stone/empty 
  Ones          1          A constant plane ﬁlled with 1 
  Turns since   8          How many turns since a move was played 
  Liberties     8          Number of liberties (empty adjacent points) 
  Capture size  8          How many opponent stones would be captured 
  Self-atari size 8        How many of own stones would be captured 
  Liberties after move 8   Number of liberties after this move is played 
  Ladder capture 1         Whether a move at this point is a successful ladder capture 
  Ladder escape 1          Whether a move at this point is a successful ladder escape 
  Sensibleness  1          Whether a move is legal and does not ﬁll its own eyes 
  Zeros         1          A constant plane ﬁlled with 0 
  Player color  1          Whether current player is black
  
  I wonder why these features are necessary, because I think a constant plane contains no information and it makes the the network larger and consequently harder to train.
  What's more, I don't understand the sharp sign here. Does it mean "the number"? But one number is enough to represent "the number of turns since a move was played", why eight?
  Thank you very much.
  
  **Tags:** machine-learning, alphago
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2020-03-01. Score: 3
  
  > I wonder why these features are necessary, because I think a constant plane contains no information and it makes the the network larger and consequently harder to train.
  
  In many implementations of convolutional layers, the filters do not neatly remain inside the features plane when "sliding" along it, but (conceptually) also partially go "outside" the plane (where always at least one "cell" of the filter will still be inside the plane). Intuitively, in the case of a $3\times3$ filter for example, you can imagine that we pad the input features with an extra border of size $1$, and this padding around the "real" input planes is filled with $0$s.
  
  If it's possible for all input features to also have values of $0$, it may in some situations be difficult or impossible for the neural network to distinguish the "real" $0$ inputs from the $0$ entries in the padding around the board, i.e. it may struggle to know where the game board is and where the game board ends. Having a constant input plane that's always filled with $1$s can help in this respect, because that plane can always reliably be used to distinguish "real" cells that actually exist on the game board from positions that are just outside the game board.
  
  As for the plane filled with $0$s... I have no idea why that would ever be useful. Maybe it was useful due to some peculiar implementation detail. In [this thread](https://www.reddit.com/r/MachineLearning/comments/553jez/has_deepmind_released_their_alphago_network/d878p8u/), some people hypothesise that on specific hardware it might make some computations slightly more efficient because of the layout of the data in memory -- it causes the number of channels to be divisible by $8$, which will.. maybe help? I really don't know too much about this, but I do know that on a smaller scale, [sometimes adding unused data in classes can indeed increase performance due to better layout in memory](https://software.intel.com/en-us/articles/coding-for-performance-data-alignment-and-structures). I suppose it's also possible that it was accidentally added by mistake, or "just in case" and that it doesn't really have much of a purpose. The amount of hardware that the AlphaGo team had available to them is quite insane anyway, one channel more or less probably wasn't too big of a concern for them.
  
  ---
  
  > What's more, I don't understand the sharp sign here. Does it mean "the number"? But one number is enough to represent "the number of turns since a move was played", why eight?
  
  This is explained in the "**Features for policy/value network**" paragraph in the paper. Quote from the paper:
  
  > "Each integer feature value is split into multiple $19 \times 19$" planes of binary values (one-hot encoding). For example, separate binary feature planes ares used to represent whether an intersection has $1$ liberty, $2$ liberties, $\dots$, $\geq 8$ liberties.
  
  All feature planes used were strictly binary, no feature planes were used that had integer values $> 1$. This is quite common when possible, because neural networks tend to have a much easier time learning with binary features than with integer- or real-valued features.
  
  
  ---
  
  