Memorizing Transformers
The ‘ethics’ section seems surprisingly cursory and lacking in references.“The ability to memorize large databases of facts could have potential ramifications for society, especially if those databases include sensitive personal information or copyrighted works. However, one advantage of using an external memory is that the memory can be easily cleared of all such information”That’s it? Just ‘may have ramifications’?No concern that this enables ‘Tay’-like failure modes where a system can be manipulated through input into generating particular output?Or even just grappling with whether adding ‘memory of experiences’ to a language model might open the door to creating a system that has beliefs, or opinions…? and that maybe there might be some ethical concerns with just wiping that out?