In my model I sum the output of three layers and run that sum over a tanh and another layer, like in the code below. whereas with torch 0.3 adding the three variables does not produce an increase in GPU memory usage, in 0.4 the memory usage increases on every variable addition, producing OOM after a few iterations (this step is the attention of a decoder)
How can one fix this such that adding these variables produces no increase in gpu memory usage and why does this happen on 0.4 but not in 0.3?
      processed_query = self.query_layer(query)
      processed_memory = self.memory_layer(memory)
      processed_attention_weights = self.location_layer(attention_weights)
      alignment = processed_query + processed_attention_weights + processed_memory
      alignment = self.v(F.tanh(alignment))