all evaluated lazily

tf.data doesn’t run anything until you iterate over the dataset, that part is lazy.
But if you look for examples on tensorflow.org 11 you’ll see they all set the mean/variance one way or another before using the Normalization layer 3, or any other similar layer.
You can either set them as arguments to the constructor, or use the .adapt which sets the statistics based on all the data you give it.
If you want it to to calculate the mean per batch (or an EMA across batches), Use BatchNormalization 3.