Hi,
I have tested your code and error is about your weight initialization. Actually, you have to transpose the weight you have considered or reverse sizes.
but the conventional way of initializing weight or std is using the torch.nn.init module.

Here is your code using this convention, and it works:


import torch
import torch.nn as nn

w_di = nn.Linear(68, 1024, bias= True).cuda()
indices = torch.randn(32, 6, 68).cuda()
torch.nn.init.normal_(w_di.weight)
torch.nn.init.constant_(w_di.bias, 1.0)
w_di(indices)


And this is your updated code:


import torch
import torch.nn as nn

w_di = nn.Linear(68, 1024, bias= True).cuda()
indices = torch.randn(32, 6, 68).cuda()
w_di.weight = nn.Parameter(torch.randn(34 * 2,1024).t().cuda())
w_di.bias = nn.Parameter(torch.ones(1024).cuda())  
w_di(indices)

Bests
Nik