Do you get this output running my code example?
The weight parameter as well as the gradient should be set to zero for the marked positions.
weight shouldn’t have any values in the masked region in step2.


torch.roll with some views should work:


in_features = 4
out_features = 5
mask_width = 3

x = torch.cat((
    torch.ones(mask_width),
    torch.zeros(out_features-mask_width)
))

mask = torch.cat([torch.roll(x, i) for i in range(in_features)])
mask = mask.view(in_features, out_features).t()

This is most likely not the most efficient way, but if you only have to create the mask once, it could be good enough for now.