nice! <3
Is there an easy way to sort out what is an activation layer vs other layers?
Ideally the more fine granularity the better e.g.

sequence layers
affine
activations
normalization
regularizations

etc but at the very least separate activations layers from the rest…