Hi, I also want to implement a split batch norm layer and I can decide to use which in different forward steps. I want to do something like these:
output1=model(data1)
model.switch("bn2")
output2=model(data2)
model.switch("bn1")
output3=model(data3)
...

How can I implement such feature?