Solved by tom in post #2 
              
              
                Linear layer weights are of shape out_feat x in_feat, conv weights are out_chan x in_chan x kernel_height x kernel_width, so all you need is to use channels as features and then add two dimensions to the weight: 
with torch.no_grad():
  conv_layer.weight.copy_(lin_layer[:, :, None, None])

Indexing …