Hi All,
I would like to ask your help with an implementation. I am trying to train 200 regressors on the same data ( think of these as 200 copies of the same model and same X, but different Y values to regress to for each).
My future implementation would involve aggregating these 200 predicted values. But for now, I just want to run 200 copies of models, all having the same input but different Y_true to regress to.  Also, I do not want them to share weights i.e. each should independently update its weights.
To create this, I tried instantiating multiple instances of the net as follows:
class NeuralNetwork(nn.Module):
        def __init__(self):
            super(NeuralNetwork, self).__init__()
            self.layer1 = nn.Linear(X.shape[1], 100) # 
            self.layer2 = nn.Linear(100, 10)
            self.layer3 = nn.Linear(10, 1)


        def forward(self, x):
            x = F.relu(self.layer1(x))
            x = F.relu(self.layer2(x))
            x = self.layer3(x)

            return x

models = [] #to store instances of NNs, one per regressor
losses = [] #to store regressor-wise losses
y_pred = [] #to store the regression predictions

models = []
losses = [0]*200
y_pred = [0]*200
for i in range(200):  # Initialization loop
  m = NeuralNetwork()
  optimizer = optim.Adam(m.parameters(), lr=0.005)
  mse_loss = nn.MSELoss()
  models.append(m)

for i in range(200): # Execution loop
  for epoch in range(10):
    predicted = models[i](x_train_tensor)
    y_pred[i] = predicted
    loss = mse_loss(predicted,y_train_tensor[i])
    losses[i] = loss
    loss.backward()
    optimizer.step()

I have 3 questions about this implementation:
a) Would this work? 
b) Is the execution loop the right place to call optimizer.step()?
c) How can I parallelize these regressors? I was reading up on Dataparallel module, but that seems to be for models training on different Xs itself. Could you please share some ideas on this?