Answers.Body: <p>I was trying to solve the same problem but since I am not familiar with C++ I did it in Python instead. My program also unzips and extracts the PGN files from the downloaded zip files.</p>
  
  <h2>Code:</h2>
  
  <pre><code>  import pycurl
    from io import BytesIO
    import os
    import zipfile
    from pathlib import Path
  
    c = pycurl.Curl()
  
    for i in range(start, end + 1):
  
      url = "http://theweekinchess.com/zips/twic" + str(i) + "g.zip"
  
      save_location_path = Path("/home/paluchasz/Desktop/chessbase_stuff/downloading_twic/twic" + str(i) + ".zip")
      unzipped_location_path = Path("/home/paluchasz/Desktop/chessbase_stuff/downloading_twic")
  
      with open(save_location_path, 'wb') as f:
  
          c.setopt(c.URL, url)
          c.setopt(c.WRITEDATA, f)
          c.perform()
  
      with zipfile.ZipFile(save_location_path, 'r') as myzip:
          myzip.extractall(unzipped_location_path)
  
      os.remove(save_location_path)
  
    c.close()
  </code></pre>
  
  <h2>Notes:</h2>
  
  <ul>
  <li>Firstly to find the url I just put my mouse pointer on the download pgn button on the TWIC website. I noticed that the urls have a common structure and the only thing that changes are the numbers. However, that url appeared to be incorrect as I was getting a 301 (moved permanently) error. As it turned out it moved to the same url but without the 'www' at the start.</li>
  <li>Adjust the range function depending on what files you want to download, so if you want to download from 1100 to 1250 then set start = 1100 and end = 1250</li>
  <li>The unzipped_location_path should be the path to the folder where you want the pgn files to be saved to. The save_location_path will be a path to where the zip files will be downloaded to first (I chose the same folder for this). These also need a file name at the end of the path which needs to be different for each zip. Hence the <em>str(i) + ".zip"</em> ending.</li>
  <li>It is not necessary to use <em>Path()</em> - it's use is that this script can now be run on another operating system which perhaps has the same path (Windows for instance uses backslashes instead and this path would not work)</li>
  <li>The pycurl library downloads the files. Zipfile then reads these files and extracts them. Finally <em>os.remove(save_location_path)</em> removes the no longer useful zip files.</li>
  <li>Finally, I originally downloaded CBV Chessbase files instead. However, I could not see an easy way to combine all those files together and I would have had to append to my database one file at a time. That is why I downloaded PGN instead as now I could use the cat command ($ cat * >twic1100-1250.pgn) to merge all the files into one.</li>
  </ul>
  
  <h2>Automating the process</h2>
  
  <ul>
  <li>Finally, I thought to myself: would it not be nice if the script was executed weekly and my database would be up to date from now on? To see how I adapted my code have a look on my GitHub: <a href="https://github.com/paluchasz/downloading_games_from_TWIC/blob/master/downloading_twic_games.py" rel="nofollow noreferrer">https://github.com/paluchasz/downloading_games_from_TWIC/blob/master/downloading_twic_games.py</a></li>
  <li>I then set up a cron job in the command line to execute the script every Tuesday.</li>
  </ul>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-11-02T03:02:53.180Z
Answers.Text: Thanks for sharing! Does this script work on Windows or Linux or both? Paths look like Linux, but you mention ChessBase:)
Answers.UserDisplayName: Joe
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-11-03T13:35:45.720Z
Answers.Text: I think it should work on both although I only tried on Linux! The point of Path() is so it works on both but you could just use the windows path with backslashes instead and then it should work. I am using Linux but I have a Windows in Virtual Box - that's why I mentioned ChessBase. I have a shared folder between the two systems and that's where I save the pgn files!
Answers.UserDisplayName: Szymon Palucha
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-11-01T15:49:05.843Z
Answers.LastActivityDate: 2018-11-01T15:49:05.843Z
Answers.UserDisplayName: Szymon Palucha
Answers.Body: <p>An easy solution in C++:</p>
  
  <pre><code>    #include &lt;urlmon.h&gt;
  
      #pragma comment(lib, "urlmon.lib")
  
      const int firstnumber = 920, lastnumber = 1104;
      const char * const Spath = "http://www.theweekinchess.com/zips/twic"; // source path
      const char * const Dpath = "C:\\Users\\&lt;username&gt;\\Downloads\\twic\\"; // destination path
  
      void main()
      {
          int x;
  
          for (x = firstnumber; x &lt;= lastnumber; ++x)
              URLDownloadToFile(nullptr, (Spath + std::to_string(x) \
      + "g.zip").c_str(), (Dpath + std::to_string(x) + ".zip").c_str(), 0, nullptr);
  
  
      }
  </code></pre>
  
  <p>A powershell unzip script:</p>
  
  <pre><code>$shell = new-object -com shell.application
  
  for ($x = 920; $x -le 1103; $x+=1) {
  
      $zip = $shell.NameSpace(“C:\Users\&lt;username&gt;\Downloads\twic\$x.zip”)
  
      foreach($item in $zip.items())
      {
          $shell.Namespace(“C:\Users\&lt;username&gt;\Downloads\twic”).copyhere($item)
      }
  
  }
  </code></pre>
  
  <p>A simple command line pgn file merge: <code>copy *.pgn twic.pgn</code></p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2016-01-01T23:46:14.100Z
Answers.LastActivityDate: 2016-01-02T00:03:40.247Z
Answers.LastEditDate: 2016-01-02T00:03:40.247Z
Answers.UserDisplayName: limits
Answers.Body: <p>Here is your requested zip:  <a href="http://nlightening.com/test/2013_TWIC.zip" rel="noreferrer">2013_TWIC.zip</a></p>
  
  <p>I created a command script for this. See attached. I assumed you are running
  Windows, but the script is easily adapted to other OSes. </p>
  
  <p>For downloading, it uses curl. You can get that at <a href="http://curl.haxx.se/download.html" rel="noreferrer">http://curl.haxx.se/download.html</a>, and
  for Windows, I recommend the msi installer.</p>
  
  <p>For unzipping, I used Java's Jar tool, which is free. But use whatever you
  prefer. </p>
  
  <p>When running the script, run dos shell as Administrator in Win8 or Win7. 
  At the end of the script it puts all PGNs into one large PGN file called 2013_TWIC.pgn
  HTH.</p>
  
  <p>Click: <a href="http://nlightening.com/test/GetTwic2013Pgn.cmd" rel="noreferrer">Script to get 2013 TWIC PGN files</a> </p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2013-12-08T18:17:27.067Z
Answers.LastActivityDate: 2013-12-09T01:55:06.727Z
Answers.LastEditDate: 2013-12-09T01:55:06.727Z
Answers.UserDisplayName: ezaspi
Body: <p>I am looking for getting all games played in 2013 as quickly and painlessly as possible. A batch file to download many ziped pgn files would be non-ideal, but ok. I prefer the Week in Chess as a source due to their high DB consistency quality, but the ultimate goal is to compile a single high quality DB of all games played in the year 2013.</p>
  
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2013-12-01T03:32:50.373Z
Comments.Text: Besides a TWIC download?  If you're willing to wait + pay, the megabases that Chessbase releases in 2014 will pretty much cover TWIC for 2013 (as well as every other year before that! :))
Comments.UserDisplayName: shivsky
Tags: <database><download-games>
Title: What is the best way to get games for the multiple editions of The Week In Chess?
ThreadText: # What is the best way to get games for the multiple editions of The Week In Chess?
  
  (Asked by: username_0 on 2013-12-01)
  
  I am looking for getting all games played in 2013 as quickly and painlessly as possible. A batch file to download many ziped pgn files would be non-ideal, but ok. I prefer the Week in Chess as a source due to their high DB consistency quality, but the ultimate goal is to compile a single high quality DB of all games played in the year 2013.
  
  **Tags:** database, download-games
  
  > **Comments:**
  > **username_1**: Besides a TWIC download?  If you're willing to wait + pay, the megabases that Chessbase releases in 2014 will pretty much cover TWIC for 2013 (as well as every other year before that! :))  
  
  ---
  
  ## 3 answers
  
  ### Answer by username_4 on 2013-12-08. Score: 4 | Accepted answer
  
  Here is your requested zip: [2013\_TWIC.zip](http://nlightening.com/test/2013_TWIC.zip)
  
  I created a command script for this. See attached. I assumed you are running
  Windows, but the script is easily adapted to other OSes.
  
  For downloading, it uses curl. You can get that at <http://curl.haxx.se/download.html>, and
  for Windows, I recommend the msi installer.
  
  For unzipping, I used Java's Jar tool, which is free. But use whatever you
  prefer.
  
  When running the script, run dos shell as Administrator in Win8 or Win7.
  At the end of the script it puts all PGNs into one large PGN file called 2013\_TWIC.pgn
  HTH.
  
  Click: [Script to get 2013 TWIC PGN files](http://nlightening.com/test/GetTwic2013Pgn.cmd)
  
  
  ---
  
  ### Answer by username_2 on 2018-11-01. Score: 3
  
  I was trying to solve the same problem but since I am not familiar with C++ I did it in Python instead. My program also unzips and extracts the PGN files from the downloaded zip files.
  Code:
    import pycurl
    from io import BytesIO
    import os
    import zipfile
    from pathlib import Path
  
    c = pycurl.Curl()
  
    for i in range(start, end + 1):
  
      url = "http://theweekinchess.com/zips/twic" + str(i) + "g.zip"
  
      save_location_path = Path("/home/paluchasz/Desktop/chessbase_stuff/downloading_twic/twic" + str(i) + ".zip")
      unzipped_location_path = Path("/home/paluchasz/Desktop/chessbase_stuff/downloading_twic")
  
      with open(save_location_path, 'wb') as f:
  
          c.setopt(c.URL, url)
          c.setopt(c.WRITEDATA, f)
          c.perform()
  
      with zipfile.ZipFile(save_location_path, 'r') as myzip:
          myzip.extractall(unzipped_location_path)
  
      os.remove(save_location_path)
  
    c.close()
  
  Notes:
  
  Firstly to find the url I just put my mouse pointer on the download pgn button on the TWIC website. I noticed that the urls have a common structure and the only thing that changes are the numbers. However, that url appeared to be incorrect as I was getting a 301 (moved permanently) error. As it turned out it moved to the same url but without the 'www' at the start.
  Adjust the range function depending on what files you want to download, so if you want to download from 1100 to 1250 then set start = 1100 and end = 1250
  The unzipped_location_path should be the path to the folder where you want the pgn files to be saved to. The save_location_path will be a path to where the zip files will be downloaded to first (I chose the same folder for this). These also need a file name at the end of the path which needs to be different for each zip. Hence the str(i) + ".zip" ending.
  It is not necessary to use Path() - it's use is that this script can now be run on another operating system which perhaps has the same path (Windows for instance uses backslashes instead and this path would not work)
  The pycurl library downloads the files. Zipfile then reads these files and extracts them. Finally os.remove(save_location_path) removes the no longer useful zip files.
  Finally, I originally downloaded CBV Chessbase files instead. However, I could not see an easy way to combine all those files together and I would have had to append to my database one file at a time. That is why I downloaded PGN instead as now I could use the cat command ($ cat * >twic1100-1250.pgn) to merge all the files into one.
  
  Automating the process
  
  Finally, I thought to myself: would it not be nice if the script was executed weekly and my database would be up to date from now on? To see how I adapted my code have a look on my GitHub: https://github.com/paluchasz/downloading_games_from_TWIC/blob/master/downloading_twic_games.py
  I then set up a cron job in the command line to execute the script every Tuesday.
  
  > **Comments:**
  > **username_0**: Thanks for sharing! Does this script work on Windows or Linux or both? Paths look like Linux, but you mention ChessBase:)  
  > **username_2**: I think it should work on both although I only tried on Linux! The point of Path() is so it works on both but you could just use the windows path with backslashes instead and then it should work. I am using Linux but I have a Windows in Virtual Box - that's why I mentioned ChessBase. I have a shared folder between the two systems and that's where I save the pgn files!  
  
  ---
  
  ### Answer by username_3 on 2016-01-01. Score: 3
  
  An easy solution in C++:
      #include 
  
      #pragma comment(lib, "urlmon.lib")
  
      const int firstnumber = 920, lastnumber = 1104;
      const char * const Spath = "http://www.theweekinchess.com/zips/twic"; // source path
      const char * const Dpath = "C:\\Users\\\\Downloads\\twic\\"; // destination path
  
      void main()
      {
          int x;
  
          for (x = firstnumber; x <= lastnumber; ++x)
              URLDownloadToFile(nullptr, (Spath + std::to_string(x) \
      + "g.zip").c_str(), (Dpath + std::to_string(x) + ".zip").c_str(), 0, nullptr);
  
  
      }
  
  A powershell unzip script:
  $shell = new-object -com shell.application
  
  for ($x = 920; $x -le 1103; $x+=1) {
  
      $zip = $shell.NameSpace(“C:\Users\\Downloads\twic\$x.zip”)
  
      foreach($item in $zip.items())
      {
          $shell.Namespace(“C:\Users\\Downloads\twic”).copyhere($item)
      }
  
  }
  
  A simple command line pgn file merge: copy *.pgn twic.pgn
  
  
  ---
  
  