text_combined: davies alex awycctsptccom original message davies alex sent 10 october 2007 1828 qtog3epoawmhperlorg cc oktrhwhrdancojp subject patch faster unicodexs decode_xs heres patch speeds reading unicode data via encodeunicodes decode_xs function test script simply reads utf16le data runs roughly half time works replacing previous grow result buffer character method allocating maximium possible length buffer begin patch uses maximum possible length c ulen 1 utf8_maxlen massively pessimistic ie smaller estimate possible nb method assumes decode_xs called arbitrary length input buffer ie arent potential going hit heavy handed utf8_maxlen multiplier requiring large allocation svgrow might fail previous code would however encodingxs always packages tobedecoded buffers 1024 byte chunks think ok unfortunately isnt case gets called via utf8 encodedecodeutf16le utf16_data patch needs smarter allocation avoid wasted memory maybe allocate single buffer front patch big enough upper limit think like 1ulensizeutf8_maxlen end copy result onlyasbigasnecesary buffer end know large free temporary holding buffer wed 06 aug 2008 012320 0400 patch faster unicodexs decode_xs