CUDA out of memory (ALBERT)!!
# ❓ Questions & Help
While using `albert-base-v2` to train my model, I got the following problem:

```
#first call
outputs = self.albert(input_ids,
                              attention_mask=attention_mask,
                              token_type_ids=token_type_ids,
                              position_ids=position_ids,
                              head_mask=head_mask,
                              inputs_embeds=inputs_embeds)

sequence_output = outputs[0]

context_mask = token_type_ids * attention_mask
question_mask = ((1 - context_mask) * attention_mask)

#second call
question, _ = self.albert(input_ids, attention_mask=question_mask, token_type_ids=token_type_ids)

#third call
context, _ = self.albert(input_ids, attention_mask=context_mask, token_type_ids=token_type_ids)
```

While calling `self.albert()` thrice, the memory it consumes will multiply by 3.
So that I must change my batch_size to 4, it's so bad!
Is it a BUG or a feature?
Even `albert-base` so large?

Hi there. Questions like this should be asked on the forum. In your code example, you are using different variable names for each of your three call, so it's logical to get more memory consumption. Python will only release the memory if you reuse the same variable name.<|||||>Hi,friends.
`you are using different variable names for each of your three call`
what do you mean? 
My problem is that:
when I first call `outputs = self.albert()` , the memory is almost 4G
And I second call `question, _ = self.albert()` , the memory increase to almost 8G.
But the variable `question` should not take up so much memory.<|||||>> Hi there. Questions like this should be asked on the forum. In your code example, you are using different variable names for each of your three call, so it's logical to get more memory consumption. Python will only release the memory if you reuse the same variable name.

Hi,friends.
`you are using different variable names for each of your three call`
what do you mean? 
My problem is that:
when I first call `outputs = self.albert()` , the memory is almost 4G
And I second call `question, _ = self.albert()` , the memory increase to almost 8G.
But the variable `question` should not take up so much memory.<|||||>Agree with @sgugger. Question like this should be asked on https://discuss.huggingface.co

We are tying to keep the issues for bug reports and new features/model requests.

Closing this for now.