Discovering latent knowledge in language models without supervision
Back when I was messing around with LSTM models I was interested in training classifiers to find parts of the internal state that light up when the model is writing a proper name or something like that.<p>Nice to see people are doing similar things w&#x2F; transformers.<p>Truth, though, is a bit problematic.  The very existence of the word makes it possible for &quot;the truth is out there&quot; to be part of the opening of the TV series the <i>X Files</i>,  see <i>Truth Social</i>.  I&#x27;m sure there is a &quot;truthy&quot; neuron in there somewhere,  but one aspect (not the only aspect) of truth is the evaluation of logical formulae (consider the evidence and reasoning process used in court) and when you can do that you run into the problems that Gödel warned you about -- regardless of what kind of technology you used.