Hey everyone,
I have a minor issue and I am not sure if it is a bug or I am simply not understanding something. But I noticed that in my project I was getting some strange results for certain configurations and error messages not popping up when expected.
To boil it down, I am allowed to multiply matrices of incompatible sizes so long as it takes place on the gpu.
When running the following:
input = torch.randn(5, 10)
network = nn.Linear(3, 3)

output = network(input) ## This should fail

I get the expected error:
RuntimeError: mat1 and mat2 shapes cannot be multiplied (5x10 and 3x3)

However, the following runs without errors:
input = torch.randn(5, 10).cuda()
network = nn.Linear(3, 3)
network.cuda()

output = network(input) ## This should fail

It gives me an output with shape (5,3).
In fact I can run with input shape (5, 8793289) and it will still work and I will still get a output with shape (5,3).
So what is happening here?