Visualizing Neural Network Imagination
<|reference_start|>Visualizing Neural Network Imagination: In certain situations, neural networks will represent environment states in their hidden activations. Our goal is to visualize what environment states the networks are representing. We experiment with a recurrent neural network (RNN) architecture with a decoder network at the end. After training, we apply the decoder to the intermediate representations of the network to visualize what they represent. We define a quantitative interpretability metric and use it to demonstrate that hidden states can be highly interpretable on a simple task. We also develop autoencoder and adversarial techniques and show that benefit interpretability.<|reference_end|>
@article{wichers2024visualizing,
    title={Visualizing Neural Network Imagination},
    author={Nevan Wichers, Victor Tao, Riccardo Volpato, Fazl Barez},
    journal={arXiv preprint arXiv:2405.06409},
    year={2024},
    archivePrefix={arXiv},
    eprint={2405.06409},
    primaryClass={cs.LG cs.AI}
}