Parsing baseball files in Rust instead of Python for an 8x speedup
The parse code at <a href="https:&#x2F;&#x2F;github.com&#x2F;gregstoll&#x2F;baseballstats&#x2F;blob&#x2F;443dbefe1d9bb5de0f01ab0c7f09ed1e5fdcb5eb&#x2F;parseretrosheet.py" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;gregstoll&#x2F;baseballstats&#x2F;blob&#x2F;443dbefe1d9b...</a> shows a few ways the Python code could be sped up:<p><pre><code>        if (not doneParsingEvent):
            if (batterEvent.startswith(&#x27;K&#x27;)):
                runnerDests[&#x27;B&#x27;] = 0
                runnersDefaultStayStill = True
                if (batterEvent.startswith(&#x27;K+&#x27;) or batterEvent.startswith(&#x27;K23+&#x27;)):
                    if batterEvent.startswith(&#x27;K+&#x27;):
</code></pre>
The &quot;not doneParsingEvent&quot; is used to emulate a &quot;continue&quot; statement. As is, every line can trigger a large number if &quot;if (not doneParsing):&quot; tests.<p>Python&#x27;s s.startswith() methods are very attractive, but slow compared to the error-prone:<p><pre><code>    if batterEvent[0] == &quot;K&quot;:
      ...
      if batterEvent[:2] == &quot;K+&quot;:
         ..
</code></pre>
Here are the timings:<p><pre><code>    % python -m timeit -s &#x27;s = &quot;Knights&quot;&#x27; &#x27;s.startswith(&quot;K&quot;)&#x27;
    2000000 loops, best of 5: 119 nsec per loop
    % python -m timeit -s &#x27;s = &quot;Knights&quot;&#x27; &#x27;s[:1] == &quot;K&quot;&#x27;
    5000000 loops, best of 5: 76.5 nsec per loop
    % python -m timeit -s &#x27;s = &quot;Knights&quot;&#x27; &#x27;s[0] == &quot;K&quot;&#x27;
    5000000 loops, best of 5: 41.4 nsec per loop
</code></pre>
There are a number of uses of:<p><pre><code>    simpleHitMatch = getRe(r&quot;^([SDTH])(?:\d|&#x2F;)&quot;).match(batterEvent)
</code></pre>
where &quot;getRe()&quot; implements a simple regex cache. However 1) re.match() also implements a simple regex cache, and 2) I typically implement this as a per-module variable, one per regex:<p><pre><code>    _simple_hit_pattern_matcher = re.compile(r&quot;^([SDTH])(?:\d|&#x2F;)&quot;).match
</code></pre>
which is then used as:<p><pre><code>    simpleHitMatch = _simple_hit_pattern_matcher(batterEvent)
</code></pre>
This would reduce the call overhead by removing the double cache table lookup.<p>Bear in mind that I don&#x27;t know where the time is spent (so I may be looking in the wrong place) and I don&#x27;t think these changes could get an 8x performance improvement.