In these lines of code:
latent = torch.empty(x.shape[0], self.hidden_features, 1)
    
for feature in range(x.shape[-1]):
    latent = torch.cat((latent, self.encoder(x[:,:,feature]).unsqueeze(2)),dim=2)

you are appending an empty and thus uninitialized tensor to itself. Since you are not initializing the values of the tensor, it might contain invalid values such as Infs/NaNs etc.
I would recommend to append the output of the encoder to a list and use torch.stack afterwards.