I would recommend to start with this tutorial 2 and try to replace the Keras layers with their PyTorch equivalents:


Dense → nn.Linear


Actiation('sigmoid') → nn.Sigmoid (or torch.sigmoid in the forward method)

Dropout → nn.Dropout


Multiply → x = x * att_weights in the forward method (or a custom nn.Module, which doesn’t seem to be necessary)

Lambda → I guess you could just use x = x.mean(dim=1) in the forward method