Answers.Body: <p>My notes on how games are encoded in sg4 format in case anybody else runs into this issue.</p>
  <p>Games begin with 0000 and end with 0f.</p>
  <p>Other than queens moving diagonally, each move is encoded as one byte.  The first four bits indicate which piece (for the side to move) is moving, the last four bits describe the move.  The convention is different for each piece.</p>
  <p>These are the codes for each piece at the beginning of the game:
  0 = king,
  1 = queen's rook,
  2 = queen's knight,
  3 = queen's bishop,
  4 = queen,
  5 = king's bishop,
  6 = king's knight,
  7 = king's rook,
  8 = a-pawn,
  9 = b-pawn,
  a = c-pawn,
  b = d-pawn,
  c = e-pawn,
  d = f-pawn,
  e = g-pawn,
  f = h-pawn.</p>
  <p>Tricky thing to note here is that whenever a piece is captured, its code is reassigned.  The first captured piece (for each color) has its code reassigned to whatever piece is code f, the next to whatever piece is code e, and so on.  EX: 1. e4 e5 2. f4 exf4, White's f4 pawn (d) is captured, now White's h2 pawn (originally f) is encoded as d.</p>
  <p>Moves for each piece: (Relative directions given the viewer has the a1 square in the bottom-left)</p>
  <ul>
  <li>Pawns: f = 2 squares forward, 1 = 1 square forward, 2 = capture along the positive diagonal (i.e. e4 to f5, or f5 to e4), 0 = capture along the negative diagonal.  I leave promotions as an exercise to the reader, but there are 12 remaining hex values and 12 possible ways to promote (4 different pieces to promote to, and 3 different directions to move to the promotion square).</li>
  <li>Knights: 8 = Up 2 Right 1, 6 = Up 1 Right 2, 4 = Down 1 Right 2, 2 = Down 2 Right 1, 1 = Down 2 Left 1, 3 = Down 1 Left 2, 5 = Up 1 Left 2, 7 = Up 2 Left 1.</li>
  <li>Bishops: 0 through 7 indicates that the bishop moves to the specified file (a through h) along the positive diagonal, e.g. if the bishop is on d4, code 5 means the bishop moves to the f-file, so Bf6.  Codes 8 through f indicate that the bishop moves to the specified file (a through h again) along the negative diagonal, e.g. if a bishop is on b7, code e means Bg2.</li>
  <li>Rooks: Codes 0 through 7 mean horizontal movement to the specified file.  If a rook is on d3, code 7 means Rh3.  Codes 8 through f mean vertical movement to the specified rank.  If a rook is on d3, code b means Rd4.</li>
  <li>Queens, orthogonal: Exactly the same as rooks.  However, a byte indicating a queen moving horizontally to the file she is already on (e.g. code 3 if the queen is already on the d-file) indicates instead that this is the first byte of a two-byte diagonal queen move, and the entire following byte describes the movement.</li>
  <li>Queens, diagonal: This is an absolute coordinate system that exactly specifies the destination square.  The first four bits will be a value from 4-7 specifying which group of two ranks contains the destination square (4 means the first and second ranks, 5 the third and fourth, etc).  The last four bits will be a value from 0-f indicating which square of the 16 selected is the destination, starting from the bottom-left to the bottom-right, then the top-left to the top right.  So 58 means the queen moves to a4.</li>
  <li>Kings: 1 = Down Left, 2 = Down, 3 = Down Right, 4 = Left, 5 = Right, 6 = Up Left, 7 = Up, 8 = Up Right, 9 = 0-0, a = 0-0-0.</li>
  </ul>
  <p>Variations: The byte 0d indicates the beginning of a variation, 0e indicates the end of a variation.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-06-10T07:55:25.660Z
Answers.LastActivityDate: 2024-07-20T23:34:05.320Z
Answers.LastEditDate: 2024-07-20T23:34:05.320Z
Answers.UserDisplayName: KnightFork
Answers.Body: <p>I am sure it is <em>possible</em>. The information is there (well, the <em>game</em> information is there - it wouldn't know who played which game without the index, though) and everything is open source. But if nobody has created such a recovery tool, it would require programming.</p>
  <p>I tried (with copies, of course) deleting one of my .si4 files while leaving the .sg4 and .sn4, to replicate your situation. I then took a totally separate .si4 and renamed it to match the deleted file, to try to trick the program. The effectiveness was... slightly better than zero. A very few games were able to load. (Some required double-clicking on the game, clicking &quot;OK&quot; on the error message, and double-clicking again.) You might be able to use this to at least get a game or three out of the database without learning to code.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-06-08T04:58:44.787Z
Answers.Text: I appreciate your input - I figured out that I had a total of 17 games in the database only and attempted to use a blank .si4 file containing 17 games to match to no avail; I also tried to use the large example database included with SCID which got my opponent names to load correctly but unfortunately nothing else. Concerning my coding knowledge, I have practically zero so that is not a likely avenue I can pursue. If someone is able to develop a tool, I am willing to pay for your time and effort. Otherwise, I think it's gone for good and that's ok; it's a good life lesson regardless.
Answers.UserDisplayName: huskywithahat
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-06-08T02:18:16.710Z
Answers.LastActivityDate: 2024-06-08T02:18:16.710Z
Answers.UserDisplayName: D M
Body: <p>I am trying to recover a scid database from 2019, but only have the respective .sg4 and .sn4 files left and am missing the .si4 file. From my understanding, the actual game data is stored in the .sg4 file, which is really what I care about. I switched laptops many years ago, and being much younger and careless at the time I did not check if I had all the appropriate files before discarding the old laptop. This has been on my mind for quite some time, and I was wondering if it was possible at all to still recover the game data stored in the .sg4 file. I am desperate to recover the database as it contains the games from the tournament where I won my FIDE Master title.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-06-10T07:41:06.210Z
Comments.Text: I'm willing to take a crack at coding a sg4 -> pgn converter.  I've installed SCID and screwed around with a few test databases to see whether I can crack the encoding and it totally looks doable.  Are you willing to upload your sg4 and sn4 files to a Google Drive?  Unfortunately StackExchange doesn't seem to have a DM function; if you're on Lichess or chess.com I'm happy to comment my username and we can DM that way.
Comments.UserDisplayName: KnightFork
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-06-10T21:38:29.077Z
Comments.Text: Of course, thank you so much I really appreciate your effort. The google drive of the files and my chess.com are as follows:
  
  https://drive.google.com/drive/u/0/folders/1U_xK3czq0vpvFwmfVJ7742Jt--mQM4pS
  https://www.chess.com/member/huskywithahat
Comments.UserDisplayName: huskywithahat
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-07-21T06:03:42.120Z
Comments.Text: if they're short enough, you can post a hexdump here
Comments.UserDisplayName: qwr
Tags: <scid><scidb>
Title: Conversion of Scid Datbase to PGN with only .sg4 and .sn4 files, missing .si4 file
ThreadText: # Conversion of Scid Datbase to PGN with only .sg4 and .sn4 files, missing .si4 file
  
  (Asked by: username_0 on 2024-06-07)
  
  I am trying to recover a scid database from 2019, but only have the respective .sg4 and .sn4 files left and am missing the .si4 file. From my understanding, the actual game data is stored in the .sg4 file, which is really what I care about. I switched laptops many years ago, and being much younger and careless at the time I did not check if I had all the appropriate files before discarding the old laptop. This has been on my mind for quite some time, and I was wondering if it was possible at all to still recover the game data stored in the .sg4 file. I am desperate to recover the database as it contains the games from the tournament where I won my FIDE Master title.
  
  **Tags:** scid, scidb
  
  > **Comments:**
  > **username_1**: I'm willing to take a crack at coding a sg4 -> pgn converter.  I've installed SCID and screwed around with a few test databases to see whether I can crack the encoding and it totally looks doable.  Are you willing to upload your sg4 and sn4 files to a Google Drive?  Unfortunately StackExchange doesn't seem to have a DM function; if you're on Lichess or chess.com I'm happy to comment my username and we can DM that way.  
  > **username_0**: Of course, thank you so much I really appreciate your effort. The google drive of the files and my chess.com are as follows:  https://drive.google.com/drive/u/0/folders/1U_xK3czq0vpvFwmfVJ7742Jt--mQM4pS https://www.chess.com/member/username_0  
  > **username_2**: if they're short enough, you can post a hexdump here  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2024-06-10. Score: 1 | Accepted answer
  
  My notes on how games are encoded in sg4 format in case anybody else runs into this issue.
  
  Games begin with 0000 and end with 0f.
  
  Other than queens moving diagonally, each move is encoded as one byte. The first four bits indicate which piece (for the side to move) is moving, the last four bits describe the move. The convention is different for each piece.
  
  These are the codes for each piece at the beginning of the game:
  0 = king,
  1 = queen's rook,
  2 = queen's knight,
  3 = queen's bishop,
  4 = queen,
  5 = king's bishop,
  6 = king's knight,
  7 = king's rook,
  8 = a-pawn,
  9 = b-pawn,
  a = c-pawn,
  b = d-pawn,
  c = e-pawn,
  d = f-pawn,
  e = g-pawn,
  f = h-pawn.
  
  Tricky thing to note here is that whenever a piece is captured, its code is reassigned. The first captured piece (for each color) has its code reassigned to whatever piece is code f, the next to whatever piece is code e, and so on. EX: 1. e4 e5 2. f4 exf4, White's f4 pawn (d) is captured, now White's h2 pawn (originally f) is encoded as d.
  
  Moves for each piece: (Relative directions given the viewer has the a1 square in the bottom-left)
  
  * Pawns: f = 2 squares forward, 1 = 1 square forward, 2 = capture along the positive diagonal (i.e. e4 to f5, or f5 to e4), 0 = capture along the negative diagonal. I leave promotions as an exercise to the reader, but there are 12 remaining hex values and 12 possible ways to promote (4 different pieces to promote to, and 3 different directions to move to the promotion square).
  * Knights: 8 = Up 2 Right 1, 6 = Up 1 Right 2, 4 = Down 1 Right 2, 2 = Down 2 Right 1, 1 = Down 2 Left 1, 3 = Down 1 Left 2, 5 = Up 1 Left 2, 7 = Up 2 Left 1.
  * Bishops: 0 through 7 indicates that the bishop moves to the specified file (a through h) along the positive diagonal, e.g. if the bishop is on d4, code 5 means the bishop moves to the f-file, so Bf6. Codes 8 through f indicate that the bishop moves to the specified file (a through h again) along the negative diagonal, e.g. if a bishop is on b7, code e means Bg2.
  * Rooks: Codes 0 through 7 mean horizontal movement to the specified file. If a rook is on d3, code 7 means Rh3. Codes 8 through f mean vertical movement to the specified rank. If a rook is on d3, code b means Rd4.
  * Queens, orthogonal: Exactly the same as rooks. However, a byte indicating a queen moving horizontally to the file she is already on (e.g. code 3 if the queen is already on the d-file) indicates instead that this is the first byte of a two-byte diagonal queen move, and the entire following byte describes the movement.
  * Queens, diagonal: This is an absolute coordinate system that exactly specifies the destination square. The first four bits will be a value from 4-7 specifying which group of two ranks contains the destination square (4 means the first and second ranks, 5 the third and fourth, etc). The last four bits will be a value from 0-f indicating which square of the 16 selected is the destination, starting from the bottom-left to the bottom-right, then the top-left to the top right. So 58 means the queen moves to a4.
  * Kings: 1 = Down Left, 2 = Down, 3 = Down Right, 4 = Left, 5 = Right, 6 = Up Left, 7 = Up, 8 = Up Right, 9 = 0-0, a = 0-0-0.
  
  Variations: The byte 0d indicates the beginning of a variation, 0e indicates the end of a variation.
  
  
  ---
  
  ### Answer by username_3 on 2024-06-08. Score: 0
  
  I am sure it is *possible*. The information is there (well, the *game* information is there - it wouldn't know who played which game without the index, though) and everything is open source. But if nobody has created such a recovery tool, it would require programming.
  
  I tried (with copies, of course) deleting one of my .si4 files while leaving the .sg4 and .sn4, to replicate your situation. I then took a totally separate .si4 and renamed it to match the deleted file, to try to trick the program. The effectiveness was... slightly better than zero. A very few games were able to load. (Some required double-clicking on the game, clicking "OK" on the error message, and double-clicking again.) You might be able to use this to at least get a game or three out of the database without learning to code.
  
  > **Comments:**
  > **username_0**: I appreciate your input - I figured out that I had a total of 17 games in the database only and attempted to use a blank .si4 file containing 17 games to match to no avail; I also tried to use the large example database included with SCID which got my opponent names to load correctly but unfortunately nothing else. Concerning my coding knowledge, I have practically zero so that is not a likely avenue I can pursue. If someone is able to develop a tool, I am willing to pay for your time and effort. Otherwise, I think it's gone for good and that's ok; it's a good life lesson regardless.  
  
  ---
  
  