marvin humphrey lehdcxrectangularcom nov 10 2007 807 sean orourke wrote problem works 588 unimaginably slow blead regex engine insists copying target string preserve 1 way promise perl wont touch match target similar problem regexbased tokenizer works matching utf8 string accumulating start points end points token measured unicode characters top string slicing string individual tokens matches complete end choice write xs sub basically custom mg uses semidocumented perl internals capturing offsets via 0 0 didnt work count utf8 chars top string every time theyre accessed thats unacceptably slow long strings modified version xs sub closer intended purpose attached maybe ought module something like maybe one already exists marvin humphrey rectangular research httpwwwrectangularcom thu 07 aug 2008 130013 0800 possible avoid slowdown large mg