  It is important that consumers and regulators can verify the provenance of
large neural models to evaluate their capabilities and risks. We introduce the
concept of a "Proof-of-Training-Data": any protocol that allows a model trainer
to convince a Verifier of the training data that produced a set of model
weights. Such protocols could verify the amount and kind of data and compute
used to train the model, including whether it was trained on specific harmful
or beneficial data sources. We explore efficient verification strategies for
Proof-of-Training-Data that are compatible with most current large-model
training procedures. These include a method for the model-trainer to verifiably
pre-commit to a random seed used in training, and a method that exploits
models' tendency to temporarily overfit to training data in order to detect
whether a given data-point was included in training. We show experimentally
that our verification procedures can catch a wide variety of attacks, including
all known attacks from the Proof-of-Learning literature.

Machine Learning;Cryptography and Security