Hi, I have a generator class and a class for something that is doing a simple math operation on the output of one of the intermediate outputs from the generator layer
So what this looks like is:
First few generator layers
.
.
.
math operation
.
.
other generator layers

When I create the optimizer for the generator, the parameters for my math operation object are passed as one of the generator parameters.
I want to be able to create a separate optimizer for this math operation layer and call that during training. I tried applying an instance of the math object within the generator definition but that gave me an error.
Since the generator is wrapped in a Sequential container, I can’t assign any values inside the definition -
self.mathlayer1 = x + b

gives me an error.
So how can I define the generator in a way that will let me create a different optimizer for the novel layer?