Solved by ptrblck in post #4 
              
              
                The error is thrown, if the output of the computation is needed for the backward pass, as explained here (the linked topic doesn’t exactly use your model, but might give you an idea).