Hello,
I am trying to implement my own model based on RoBERTa. My Python code looks like below.
class My_RoBERTa(nn.Module):
    def __init__(self, some_config, some_version):
        super().__init__()
        self.roberta = RobertaModel.from_pretrained(some_version)
    def forward(self, input_ids, attn_masks, labels=None):
        print(1)
        print(input_ids.size())
        print('* *')
        print(attn_masks.size())
        print('* *')
        outputs = self.roberta(input_ids, attn_masks)
        print(3)
        print(4)
        print(5)

If I instantiate this model and pass 3 examples with a length = 25, then my expectation of output is below
1
torch.Size([3,25])
* *
torch.Size([3,25])
* *
3
4
5

However, when I run it, I actually get something like below.
1
torch.Size([3,25])
* *
torch.Size([3,25])
* *
1
torch.Size([3,25])
* *
torch.Size([3,25])
* *
3
3
4
4
5
5

I think this is some kind of overwriting issue caused by the “forward” method, but I do not know how to resolve this problem. Could anyone please help me out?
Thank you so much in advance!