Hi, Marcel. It is really hard to tell anything and debug without any code snippets. Also, I can suggest finding some working implementations of some similar tasks to the task you are doing now. Maybe reading some correct implementation can help you to figure out what you are missing in your code. For instance, you can take a look at some autoencoders. I think they are trying to achieve almost the same task. All the best!