Question 1: If I have multiple separate output head layers of a network, with the one in use triggered by either a network input of the head to use or by a onehot encoded matrix (have done both implementations), is loss propagated according to the head? So for example if there were two heads, and head two were never used in the forward in a batch, would the weights exclusive to head two be unchanged?
To give a more specific context, if they are both Q value predictors for different tasks, can I simply sum the loss for each task together (calculated by a complex comparison to the relevant task rewards) does torch automatically makes weight changes proportionally? If head one’s values were perfect and head two’s values were awful, would summing the losses affect the heads equally or not?
Question 2: If i have a distribution constructed from a softmax output, such as torch.distributions.Categorical, but i sometimes want to be able to choose actions deterministically (like DDPG), how can I select the highest probability output from a distribution rather than random sampling according to probabilities?