Dear all,
we’re experimenting with Transformers for deep-learning the translation from Protein to DNA sequences. Here’s an excerpt of our code:
##
## ITERATOR
##
BATCH_SIZE = 10
train_iterator, test_iterator = BucketIterator.splits(
    (train_data, test_data), 
    batch_size = BATCH_SIZE,
    device = device, sort = False)

##
## TRANSFORMER
##
model = nn.Transformer(d_model=1, nhead=1)
criterion = torch.nn.CrossEntropyLoss()
optimizer = torch.optim.Adam(model.parameters(), lr=1)
for i, batch in enumerate(train_iterator):
    
    lbl = batch.seq_id
    src = batch.prot_seq
    trg = batch.dna_seq

    optimizer.zero_grad()
    model.train()

    output = model(src, trg)
    print(model)

However, this code produces the following error:
Traceback (most recent call last):
File “learner.py”, line 195, in 
output = model(src, trg)
File “/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/torch/nn/modules/module.py”, line 550, in call
result = self.forward(*input, **kwargs)
File “/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/torch/nn/modules/transformer.py”, line 117, in forward
print(f’src.size(2) --> {src.size(2)}’)
IndexError: Dimension out of range (expected to be in range of [-2, 1], but got 2)
src and trg both are mxn matrices, where m the length of the protein and DNA sequence respectively and n is the BATCH_SIZE. According to the documentation (https://pytorch.org/docs/master/generated/torch.nn.Transformer.html 1), the third dimensions of src and trg must be the the feature number E. What does this exactly mean? What’s the required form of src and trg? Also, how can I add E to the src and the trg matrices?
Thanks for shedding light into my confusion!