Hi @InnovArul,
Thank you for your answer, it was quite enlightening!
Yet, what are your thoughts about this approach:
class MixedApproachTiedAutoEncoder(nn.Module):
	def __init__(self, inp, out, weight):
		super().__init__()
		self.encoder = nn.Linear(inp, out, bias=False)

	def forward(self, input):
		encoded_feats = self.encoder(input)
		reconstructed_output = F.linear(encoded_feats, self.encoder.weight.t())
		return encoded_feats, reconstructed_output

I believe that by building the network this way will result in the parameters of the decoder not being differentiated when running backward because the functional module has no state but at the same time as we are performing an operation over a parameter I would expect the network graph to be updated dynamically.
What is wrong with this train of thought?
Cheers.