RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation

Though have seen a couple of posts on forum regrading this topic, still confused of what is causing the trouble.
Here’s the piece of code I’m running and got the gradients runtime error. I’m trying to calculate a representation for each cluster by doing the average, where the clusters are mwe and non-mwe.  Note, half_retrieval = int(num_retrieval/2), fw is linear layer and act is ReLU with inplace=True. Can someone help explain which part is causing the trouble?
    def forward(self, query, retrievals, sigma=4):

        #models
        PROTOTYPICAL = True
        #metrics
        EUCLIDEAN = True

        num_candidates, num_retrievals, embed_size = retrievals.size()
        half_retrieval = int(num_retrievals/2)

        #read embeddings
        q = self.embed(query)
        flat_r = self.embed(retrievals.view(-1, embed_size))
        
        #GRU
        out_query, _ = self.gru(q)  #(num_can, seq_len, hidden_size)
        out_query = self.act1(self.fw1(out_query.transpose_(2, 1)))
        out_query = self.act2(self.fw2(out_query))
        out_query = torch.squeeze(out_query)

        out_retrieval, _ = self.gru(flat_r) #(num_can*num_ret, seq_len, hidden_size)
        out_retrieval = self.act1(self.fw1(out_retrieval.transpose_(2, 1)))
        out_retrieval = self.act2(self.fw2(out_retrieval))
        out_retrieval = torch.squeeze(out_retrieval)


        if PROTOTYPICAL == True:
            out_retrieval = out_retrieval.view(num_candidates, num_retrievals, -1)
            mwe_half = out_retrieval[:, :half_retrieval, :].clone()
            non_mwe_half = out_retrieval[:, half_retrieval:, :].clone()

            rep_mwe = torch.mean(mwe_half, 1)
            rep_non_mwe = torch.mean(non_mwe_half, 1)

            if EUCLIDEAN == True:
                mwe_dist = torch.sum(torch.mul(rep_mwe, out_query), 1)
                non_mwe_dist = torch.sum(torch.mul(rep_non_mwe, out_query), 1)
                dist = torch.stack((mwe_dist, non_mwe_dist), 1)
                sim = -dist
                weights = F.softmax(sim, 1)
                attention_scores = torch.mul(weights, sim)



I found out it’s the transpose that causes the issue.