title: Regix: Extract list of templates from MediaWiki XML document
body: <p>I am attempting to use the <a href="https://vocaloid.fandom.com/wiki/Special:Statistics" rel="nofollow noreferrer">database dump</a> of the Vocaloid wiki to extract a list of every song's urls into a list. Each of the thousands of songs have their urls stored inside the template <code>{{Song box 2}}</code>. The database dump is in the format of a MediaWiki XML document.</p>
  
  <pre><code>{{Song box 2
    |color     = #AC9381; color:#445A56
    |image     = ハチ - 砂の惑星.png
    |title     = &amp;quot;'''砂の惑星'''&amp;quot;
  * Romaji: Suna no Wakusei
  * English: Sand Planet
  * Official English: DUNE
    |date      = July 21, 2017
    |views     = {{v|nn|6,947,105}} and {{v|yt|46,753,728}}
    |singers   = [[Hatsune Miku]]
    |producers = [[Hachi]] (music, lyrics, arrangement)
  * [[Minakata Laboratory]] (video)
    |links     = {{l|nn|sm31606995}} {{l|yt|AS4q9yaWJkI}}
  }}
  </code></pre>
  
  <p>To convert the MediaWiki XML document into a more workable format for extracting the urls I used the wtf_wikipedia parser to convert the document into JSON.</p>
  
  <pre class="lang-js prettyprint-override"><code>var fs = require('fs');
  const wtf = require('wtf_wikipedia');
  
  const wikiText = fs.readFileSync('vocaloid_pages_current.xml', 'utf-8');
  var data = wtf(wikiText).json();
  fs.writeFile('output.json', JSON.stringify(data, null, '\t'), 'utf8', function (err) {
      if (err) {
          console.log("An error occured while writing JSON Object to File.");
          return console.log(err);
      }
      console.log("JSON file has been saved.");
  });
  </code></pre>
  
  <p>However, the JSON returned strips the the links of the id used to determine what website the id of the video corresponds to. While it would be possible to guess the website based on how the video id is formatted, parsing the entire XML document in this manner is already very slow and not a great solution to my problem.</p>
  
  <pre><code>{
  "color": "#AC9381; color:#445A56",
  "image": "ハチ - 砂の惑星.png",
  "title": "&amp;quot;砂の惑星&amp;quot;\n* Romaji: Suna no Wakusei\n* English: Sand Planet\n* Official English: DUNE",
  "date": "July 21, 2017",
  "views": "and",
  "singers": "Hatsune Miku",
  "producers": "Hachi (music, lyrics, arrangement)\n* Minakata Laboratory (video)",
  "links": "sm31606995 AS4q9yaWJkI",
  "template": "song box 2"
  }   
  </code></pre>
  
  <p>My next idea to gather all of the song's urls is to create a list of all the text inside the <code>{{Song box 2}}</code> templates and parse the urls from there. To create a list of the text inside the <code>{{Song box 2}}</code> templates I attempted to use regular expressions. However, the <a href="https://regex101.com/r/cuP5P5/1" rel="nofollow noreferrer">regular expression I created</a> instead of selecting the all of the text inside the <code>{{Song box 2}}</code> template, will stop at the first occurrence of <code>}}</code> inside the template.</p>
  
  <p>How can I use regular expressions to create a list of the text inside the <code>{{Song box 2}}</code> templates?</p>
  
  <p><strong>Edit 1</strong></p>
  
  <p>Entering the regex pattern provided by @jhnc into the script below successfully matches 4,194 out of 7,196 what I would expect to be matches. I believe this is because some of the <code>{{Song box 2}}</code>'s have characters before <code>{{Song box 2}}</code> on the same line. <a href="https://regex101.com/r/cuP5P5/4" rel="nofollow noreferrer">See the updated regex example</a>.</p>
  
  <pre class="lang-js prettyprint-override"><code>var fs = require('fs');
  const wikiText = fs.readFileSync('vocaloid_pages_current.xml', 'utf-8');
  const regex = /^\{\{Song box 2(\s*?.*?)*?^\}\}/gm;
  const matches = [];
  let match;
  while ((match = regex.exec(wikiText)) !== null) {
    if (match.index === regex.lastIndex) {
          regex.lastIndex++;
      }
      matches.push(match[0]);
  }
  console.log(matches.length); // 4194
  </code></pre>
  
  <p>How can I also match these occurrences?</p>