Answers.Body: <p>When reading your question, my gut reaction is that your scope is too complicated for unit testing.  I recommend a quick read through the free e-book Unit Testing Succinctly.  However, I have no experience writing chess code (maybe your paradigms are different) - although I do software for a living.</p>
  
  <p>A unit test should be very simple and test a function that does 1 single thing.  Then you can combine the functions with some reasonable expectation that they will work.  For example, I would expect a unit test for each piece to determine if a particular move is legal.  A unit test for each piece to determine if it is putting the king in check.  A test for each piece to determine where it is attacking, etc.</p>
  
  <p>Testing a position seems like a very complicated unit test and would be much harder to do thoroughly.  Instead write smaller tests against smaller functions and then know that those individually work - evaluating a position is just a matter of iterating over the simple functions.</p>
  
  <p>If you want to test a position for a good (not forced) move, I think unit test's will artificially limit the long term development and strength of your chess engine... a binary result of a unit test will force your engine to make the same move every time.</p>
  
  <p>I'd also look at adding unit tests for 'most direct' path to a mate with known endgames.  I'd look to add unit tests for traversing through known openings as well.  Mid game unit tests will be much harder - maybe plugging in a position and evaluating that the engine produces a usable result (which is a binary response).</p>
  
  <p>For the question of evaluating a set of positions for your engine, you may do far better putting this question on <a href="https://stackoverflow.com/">https://stackoverflow.com/</a> with the "chess" tag.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-03-22T15:38:08.227Z
Answers.Text: Thanks for the answer. However, as you can probably tell from the comment conversation under the question, I'm not looking to test the engine algorithms. I didn't even get to those yet. I'm looking to test if my code recognizes whether a *standing position on the board* is checkmate, or a stalemate, or not, which is "just" counting legal moves and whether the king is in check.  In fact since I posted that question I went on to compose a small collection of positions to use as unit test cases. I might post that as answer. (I'm completely surprised someone offered a bounty for this, tbh.)
Answers.UserDisplayName: asibahi
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-03-22T15:45:20.403Z
Answers.Text: This doesn't answer the question.
Answers.UserDisplayName: SmallChess
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-03-22T15:47:32.563Z
Answers.Text: Furthermore, the notes about unit testing is just wrong.
Answers.UserDisplayName: SmallChess
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-03-22T17:08:10.113Z
Answers.Text: @asibahi I didn't have enough rep to post in that comment section, so I had to add an answer.  I really think you will be better served in a programming forum (stackexchange) with this question.  But on this comment... you already identified individual unit tests -> For each piece test is there a legal move.  If iterating over that returns false for each piece, you have stalemate, or checkmate if you are in check.  You don't need a vast collection of positions for that.  By testing each piece individually for its current state, you can iterate over multiple pieces to evaluate the position.
Answers.UserDisplayName: Paul
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-03-22T14:58:40.510Z
Answers.LastActivityDate: 2017-03-22T14:58:40.510Z
Answers.LastEditDate: 2017-05-23T12:37:05.403Z
Answers.UserDisplayName: Paul
Answers.Body: <p>While this is an old question, I thought that the concept presented in this blog might be useful: <a href="http://scionsoftware.com/blog/write-tests-by-playing-chess" rel="nofollow noreferrer">http://scionsoftware.com/blog/write-tests-by-playing-chess</a></p>
  
  <p>The idea is that you would play a chess game on a GUI and have a mechanism to capture the state of the chess board by serializing to a file.</p>
  
  <p>You could name those files per the test cases and feed them into whichever test method you define: IsCheckmate; IsLegal; IsDraw</p>
  
  <p>One of the main reasons you would want to use a natural UI to create these test cases, outside of ease of creation, is that enough conditions also rely on move counts: castling, en passant, draw conditions.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-02-14T21:02:53.450Z
Answers.LastActivityDate: 2018-02-14T21:02:53.450Z
Answers.UserDisplayName: SpykeBytes
Answers.Body: <p>I don't know any unit-testing database for chess engine (in general), and yes, writing exhaustive unit test is almost impossible.</p>
  
  <p>You can maybe explore alternative testing technic like property-based testing (QuickCheck in Haskell, I don't know the F# environment but it certainly exists something like that for F#), which can automatically generate a large amount of "position" and test them using property defined by you.</p>
  
  <p>Hope this will help a bit :) !</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-02-19T14:55:47.327Z
Answers.LastActivityDate: 2018-02-19T14:55:47.327Z
Answers.UserDisplayName: Sylvain Julmy
Body: <p>So this is a bit of outlier question.</p>
  
  <p>I am writing a chess library, essentially from scratch. <a href="https://codereview.stackexchange.com/questions/141425/grand-chess-domain-model-and-helper-functions">You can find much of the code here</a>, with the intention of using it for a GUI and/or an engine. (The game is Grand Chess but for the purposes of this question it doesn't really matter.)</p>
  
  <p>I am currently writing unit tests to verify that my functions work as expected. And I was wondering if there was some sort of library or database of suggested positions to test I can use and work from, categorized by whether they're Checkmate, Stalemate, Check, Legal, Illegal, etc.</p>
  
  <p>tl;dr I am looking for a list of positions to unit test my code against.</p>
  
  <p><a href="https://github.com/asibahi/Stamma/blob/master/StammaTests/PositionTests.fs" rel="nofollow noreferrer">You can find my current tests here, I add to them every few days.</a> However, I want to make sure the tests are exhaustive before I go debugging the code. (Half of them fail currently).</p>
  
  <p>Edit : to clarify : I am not looking for <em>engine</em> ("best move") tests. I am looking for <em>board representation</em> ("is this position checkmate") tests. I already have a few puzzles lined up for engine testing.</p>
  
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2016-09-27T08:03:26.250Z
Comments.Text: You're coding a variant. Standard chess dataset won't work for you. So I'm afraid you're on your own.
Comments.UserDisplayName: SmallChess
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2016-09-27T08:04:50.737Z
Comments.Text: @StudentT i am looking for something i can start from . a regular databsase would do just fine, since you know, i can edit it.
Comments.UserDisplayName: asibahi
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2016-09-27T08:07:02.307Z
Comments.Text: There're MANY such test sets, are you happy for something like perft, tactical exercises in standard chess?
Comments.UserDisplayName: SmallChess
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2016-09-27T08:08:24.647Z
Comments.Text: @StudentT I couldn't find anything via Google, thus why I am asking here. Any info would be helpful.
Comments.UserDisplayName: asibahi
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-03-18T19:25:40.223Z
Comments.Text: What you want to test has little tp do with board representation. You'll want to test board representation after making/unmaking moves or importing positions. Detecting mate/stalemate requires an evaluation function, and testing a position for legality should be a function of its own which has much more to do than checkung the board representation.
Comments.UserDisplayName: Queeg
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-03-23T12:20:29.283Z
Comments.Text: I am unsure about the FEN notations in your current tests . Why instead of 8 you have taken 10 as for an example ?                                  r8r/1nbqkmcbn1/pppppppppp/10/10/10/10/PPPPPPPPPP/1NBQKMCBN1/R8R w - 0
Comments.UserDisplayName: Seth Projnabrata
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-03-23T12:21:26.743Z
Comments.Text: @ProjnabrataSeth it's for a 10x10 variant.
Comments.UserDisplayName: asibahi
Tags: <engines><database><programming>
Title: Unit Testing a chess board representation
ThreadText: # Unit Testing a chess board representation
  
  (Asked by: username_0 on 2016-09-27)
  
  So this is a bit of outlier question.
  
  I am writing a chess library, essentially from scratch. [You can find much of the code here](https://codereview.stackexchange.com/questions/141425/grand-chess-domain-model-and-helper-functions), with the intention of using it for a GUI and/or an engine. (The game is Grand Chess but for the purposes of this question it doesn't really matter.)
  
  I am currently writing unit tests to verify that my functions work as expected. And I was wondering if there was some sort of library or database of suggested positions to test I can use and work from, categorized by whether they're Checkmate, Stalemate, Check, Legal, Illegal, etc.
  
  tl;dr I am looking for a list of positions to unit test my code against.
  
  [You can find my current tests here, I add to them every few days.](https://github.com/username_0/Stamma/blob/master/StammaTests/PositionTests.fs) However, I want to make sure the tests are exhaustive before I go debugging the code. (Half of them fail currently).
  
  Edit : to clarify : I am not looking for *engine* ("best move") tests. I am looking for *board representation* ("is this position checkmate") tests. I already have a few puzzles lined up for engine testing.
  
  **Tags:** engines, database, programming
  
  > **Comments:**
  > **username_1**: You're coding a variant. Standard chess dataset won't work for you. So I'm afraid you're on your own.  
  > **username_0**: @StudentT i am looking for something i can start from . a regular databsase would do just fine, since you know, i can edit it.  
  > **username_1**: There're MANY such test sets, are you happy for something like perft, tactical exercises in standard chess?  
  > **username_0**: @StudentT I couldn't find anything via Google, thus why I am asking here. Any info would be helpful.  
  > **username_2**: What you want to test has little tp do with board representation. You'll want to test board representation after making/unmaking moves or importing positions. Detecting mate/stalemate requires an evaluation function, and testing a position for legality should be a function of its own which has much more to do than checkung the board representation.  
  > **username_3**: I am unsure about the FEN notations in your current tests . Why instead of 8 you have taken 10 as for an example ?                                  r8r/1nbqkmcbn1/pppppppppp/10/10/10/10/PPPPPPPPPP/1NBQKMCBN1/R8R w - 0  
  > **username_0**: @ProjnabrataSeth it's for a 10x10 variant.  
  
  ---
  
  ## 3 answers
  
  ### Answer by username_4 on 2017-03-22. Score: 1
  
  When reading your question, my gut reaction is that your scope is too complicated for unit testing. I recommend a quick read through the free e-book Unit Testing Succinctly. However, I have no experience writing chess code (maybe your paradigms are different) - although I do software for a living.
  
  A unit test should be very simple and test a function that does 1 single thing. Then you can combine the functions with some reasonable expectation that they will work. For example, I would expect a unit test for each piece to determine if a particular move is legal. A unit test for each piece to determine if it is putting the king in check. A test for each piece to determine where it is attacking, etc.
  
  Testing a position seems like a very complicated unit test and would be much harder to do thoroughly. Instead write smaller tests against smaller functions and then know that those individually work - evaluating a position is just a matter of iterating over the simple functions.
  
  If you want to test a position for a good (not forced) move, I think unit test's will artificially limit the long term development and strength of your chess engine... a binary result of a unit test will force your engine to make the same move every time.
  
  I'd also look at adding unit tests for 'most direct' path to a mate with known endgames. I'd look to add unit tests for traversing through known openings as well. Mid game unit tests will be much harder - maybe plugging in a position and evaluating that the engine produces a usable result (which is a binary response).
  
  For the question of evaluating a set of positions for your engine, you may do far better putting this question on <https://stackoverflow.com/> with the "chess" tag.
  
  > **Comments:**
  > **username_0**: Thanks for the answer. However, as you can probably tell from the comment conversation under the question, I'm not looking to test the engine algorithms. I didn't even get to those yet. I'm looking to test if my code recognizes whether a *standing position on the board* is checkmate, or a stalemate, or not, which is "just" counting legal moves and whether the king is in check.  In fact since I posted that question I went on to compose a small collection of positions to use as unit test cases. I might post that as answer. (I'm completely surprised someone offered a bounty for this, tbh.)  
  > **username_1**: This doesn't answer the question.  
  > **username_1**: Furthermore, the notes about unit testing is just wrong.  
  > **username_4**: @username_0 I didn't have enough rep to post in that comment section, so I had to add an answer.  I really think you will be better served in a programming forum (stackexchange) with this question.  But on this comment... you already identified individual unit tests -> For each piece test is there a legal move.  If iterating over that returns false for each piece, you have stalemate, or checkmate if you are in check.  You don't need a vast collection of positions for that.  By testing each piece individually for its current state, you can iterate over multiple pieces to evaluate the position.  
  
  ---
  
  ### Answer by username_5 on 2018-02-14. Score: 1
  
  While this is an old question, I thought that the concept presented in this blog might be useful: <http://scionsoftware.com/blog/write-tests-by-playing-chess>
  
  The idea is that you would play a chess game on a GUI and have a mechanism to capture the state of the chess board by serializing to a file.
  
  You could name those files per the test cases and feed them into whichever test method you define: IsCheckmate; IsLegal; IsDraw
  
  One of the main reasons you would want to use a natural UI to create these test cases, outside of ease of creation, is that enough conditions also rely on move counts: castling, en passant, draw conditions.
  
  
  ---
  
  ### Answer by username_6 on 2018-02-19. Score: 0
  
  I don't know any unit-testing database for chess engine (in general), and yes, writing exhaustive unit test is almost impossible.
  
  You can maybe explore alternative testing technic like property-based testing (QuickCheck in Haskell, I don't know the F# environment but it certainly exists something like that for F#), which can automatically generate a large amount of "position" and test them using property defined by you.
  
  Hope this will help a bit :) !
  
  
  ---
  
  