Do you compute the activations beforehand / offline for all your data and store them as features / inputs for model B later? That should also work, but I was thinking you would compute them on the fly at each forward pass. But in any case, both should work.