How Many Bytes Can You Take Out Of Brain-To-Text Decoding?
<|reference_start|>How Many Bytes Can You Take Out Of Brain-To-Text Decoding?: Brain-computer interfaces have promising medical and scientific applications for aiding speech and studying the brain. In this work, we propose an information-based evaluation metric for brain-to-text decoders. Using this metric, we examine two methods to augment existing state-of-the-art continuous text decoders. We show that these methods, in concert, can improve brain decoding performance by upwards of 40% when compared to a baseline model. We further examine the informatic properties of brain-to-text decoders and show empirically that they have Zipfian power law dynamics. Finally, we provide an estimate for the idealized performance of an fMRI-based text decoder. We compare this idealized model to our current model, and use our information-based metric to quantify the main sources of decoding error. We conclude that a practical brain-to-text decoder is likely possible given further algorithmic improvements.<|reference_end|>
@article{antonello2024how,
    title={How Many Bytes Can You Take Out Of Brain-To-Text Decoding?},
    author={Richard Antonello, Nihita Sarma, Jerry Tang, Jiaru Song, and Alexander
  Huth},
    journal={arXiv preprint arXiv:2405.14055},
    year={2024},
    archivePrefix={arXiv},
    eprint={2405.14055},
    primaryClass={cs.CL cs.AI cs.ET}
}