Hi. In evaluation mode, do we need to still put the line of dropout? This is the code
import torch
import torch.nn as nn

x = torch.randn(4,4)
drop = nn.Dropout(0.5)
model = nn.Linear(4,6)
model.train()
model(drop(x))

model.eval()
d = model(drop(x))
n = model(x)
print(d == n)

This gives a matrix of 0s (all False). So which one should be correct? model(drop(x)) or model(x)? I’ve seen some GitHub codes use model(x), but then in this case how would the model scale the output at test time if drop(x) is removed at test time?