Show HN: Big List of Naughty Strings for testing user-input data
Most of what I do involves the messy world of text, and I think this is a great resource. I wish the software I depended on tested against it.<p>I can think of a few more cases that I've seen cause havoc:<p>- U+FEFF in the middle of a string (people are used to seeing it at the beginning of a string, because Microsoft, but elsewhere it may be more surprising)<p>- U+0 (it's encoded as the null byte!)<p>- U+1B (the codepoint for "escape")<p>- U+85 (Python's "codecs" module thinks this is a newline, while the "io" module and the Python 3 standard library don't)<p>- U+2028 and U+2029 (even weirder linebreaks that cause disagreement when used in JSON literals)<p>- A glyph with a million combining marks on it, but not in NFC order (do your Unicode algorithms use insertion sort?)<p>- The sequence U+100000 U+010000 (triggers a weird bug in Python 3.2 only)<p>- "Forbidden" strings that are still encodable, such as U+FFFF, U+1FFFF, and for some reason U+FDD0<p>People should also test what happens with isolated surrogate codepoints, such as U+D800. But these can't properly be encoded in UTF-8, so I guess don't put them in the BLNS. (If you put the fake UTF-8 for them in a file, the best thing for a program to do would be to give up on reading the file.)