Writing a Fast JSON Parser
I felt moved to comment on the thread. HN user burntsushi has been very nice by writing up a trick we developed here on the Hyperscan team that should be considerably faster than this system assuming there's enough data to make using SIMD pay off (this would be a lose if the sizes are small enough).<p>It's possible that a SIMD implementation (assuming that you routinely pass over enough bytes) can blow this away. I'm not sure what the cross-over point is - it depends on whether you are falling out of this code quickly or not. Obviously using a big SIMD sequence to replace a "2 cycles per byte" implementation is pretty stupid if the SIMD sequence is "0.5 cycles a byte" but the fixed cost of using the SIMD means that your costs are >=16 cycles and the typical case is small.<p>My comment follows:<p>It's also not clear that you actually need a SIMD instruction - if you've squashed your hotspot to the point that adding the complexity of per-platform SIMD specialization is only a small win, then why bother?<p>All that being said...<p>I've been really lazy in writing this up, but someone kindly did a decent write-up for us in the Rust reimplementation of our "Teddy" literal matcher. Please see <a href="https://github.com/rust-lang/regex/blob/master/src/simd_accel/teddy128.rs" rel="nofollow">https://github.com/rust-lang/regex/blob/master/src/simd_acce...</a> if interested or look up the Hyperscan project at <a href="https://github.com/01org/hyperscan" rel="nofollow">https://github.com/01org/hyperscan</a> for a few different versions of this trick.