Hi I’m very new to using PyTorch and am still wrapping my head around it all, but I was wondering given the dynamic nature of pytorch how it might be possible to add new neurons (with different parameters than the other nodes potentially) to the hidden layer partway through training. Or if I train first, add new neurons, train again, this detail is not so important… Is it as simple as modifying the tensors containing the weights?
Or would it be best to allocate the layer with all potential nodes, with some being  ‘silenced’, and make them available through training?