Have you added print statements to see where exactly its happening? I would time how long it takes to load the net(assuming you are loading both at the same time). My suspicion is that since you are trying to trace both models in the constructor, this slows things down.
There really isnt a need to trace/script the network a total of 3 times. I would remove the tracing in the constructor, and just script the whole graph at once. You only want to script the network when it will be used outside of a Python environment and scripting the network at the end like you have is all you need to achieve this.