(b) Test accuracy w.r.t. the number of epochs

Figure 4: Hyper-parameter sensitivity studies: we evaluate models in random initialization settings.
(a) Average test accuracy w.r.t. the number of GD steps. We use two epochs. (b) Average test accuracy
w.r.t. the number of epochs. We use 10 steps, with each step containing 10 images.

(a) Test accuracy w.r.t. the number of steps

(b) Test accuracy w.r.t. the number of epochs

Figure 5: Comparison between one and multiple GD steps given the same number of images. We
evaluate models in random initialization settings and use one epoch. N denotes the total number of
images per category. For multiple steps runs, each step uses 10 images in total (one per category).

Ours

Ours

k-means

97.9
93.2
96.2

w/ ﬁxed
pre-trained

Random real Optimized real

M → U
U → M
S → M

w/ random
pre-trained
95.4 ± 1.8
92.7 ± 1.4
85.2 ± 4.7
DATASET DISTILLATION