hi, I’m trying to do a Denosing Auto Encoder, and have a question - is it a bad idea to have a model without activations?
i.e. what I do is basically is:
Linear(100,1000) -> Linear(1000,1000) -> Linear(1000,100)
I also tried with Relu, i.e.:
Linear -> Relu -> Linear -> Relu -> Linear
but the one without activations seems to work better on the Validation set (converges faster and to a lower MSE loss)
So I would like to understand if what I do (Linear without ReLU) is conceptually “ok” or “wrong”