latent_space, output = VAE(input)
classification = classifier(latent_space.deatch())

Basically whatever you pass to that fully connected layer, detach() it, it breaks backprop graph.
If there were an issue while backpropagating, you should see a warning saying certain layer’s graph has already been used.