I don’t know where else to ask and I am starting to be a bit desperate. If anyone knows a place where to ask this, please tell me. Anyways, I will try it here first.
I have trained a transformer network (multiple times). The dataset consists of sequences of a plant, which grows larger (sprouting to roundabout 10 cm). The background is very simple, and there are just two lights changing direction (coming sometimes from the left and sometimes from the right). The network has the task to predict the future plant shape, so how the next frame looks like, when he gets 4 frames as an input. This works ok, but still not perfect.Now I want the network to create new plants by starting with some original images of the plants in the dataset and start prediction. But the input to the network will be replaced with the output of the network, which was produced before. So it only predicts on already predicted frames.But by doing so, after like 30 images, some fragments from before are starting to evolve to unwanted things (for example, a second plant). And the plant that had to be predicted also becomes strange.
Has anyone experienced something similar? Or with other convolutional neural networks that produced strange output when predicting already predicted things?