But you will see the same problem if you change the input type.
In general, you want to do:
tmp = torch.ones(input.size(0), output.size(2), dtype=input.dtype, device=input.device)

to make sure you get a Tensor of the same type and same device as input.
If you have the same size (not the case for this example), you can do tmp = torch.ones_like(input) to get the same size/dtype/device.