you have to distinguish python statements that use parameters and initialization actions. the later should be done without gradient tracking, inside
“with torch.no_grad(): …” context; presense of in-place ops doesn’t matter in no_grad mode.
yes, attributes, but tensors should be wrapped in nn.Parameter, to “register” them (for optimizer and other enumerators). basically, the fact that you have “non deep” model is not relevant here, you can construct things the same way as tutorial DL models.