Hello, thanks for reply


I tried to develop the model using Aladdin Persson and Pytorch examples (Word level language model), but both were not designed for my purpose, so I tried to play with their mixture.


I tried overfitting with 2 sentence and 5000 epochs but it’s same


I already random masking


def masking(src):
    mask_idx = random.randint(1, 10)
    src[mask_idx] = 1
    return src

Can be my model isn’t good for purpose?