I have a network with four subnetworks: A_model, B_model, C_model, D_model
During forwarding,
  out_A = A_model(input) 
  out_B = B_model(out_A)
  out_C = C_model(out_A)
  out_D = D_model(out_B, out_C)

B_model and C_model both take out_A as input, but they run independently.
To my understanding, PyTorch will run these lines serially, right? If B_model and C_model can run in parallel (e.g., multi-threading), will it save a lot of time?