Can you give more specific details on how do you want to choose the subset of weights? Randomly or other criteria? I assume the purpose of this is to help avoid overfitting problems.

If you want to do it randomly, then why not using the Dropout? This will also help with the overfitting problem.
In theory, you can also use L1 regression which will result in sparse weights, so some of the network weights will become zero.




 mattinjersey:

This will save computational time and memory.


If the only reason doing this is for computational time, then why not reducing your model size?