Your new use case seems to use the penultimate activation tensors as an extracted feature.
While your approach would return the feature tensor before the pooling layer (which will thus be bigger), my proposed approach would apply the pooling and thus yield a smaller activation.
I don’t know, how your model (or overall use case) will use this tensor, so I cannot say that mu proposed approach is the “right one” for your use case.



 John_J_Watson:

Also, what am I looking for in the results? Do you mean to say, I should measure accuracy in the two cases (with and withut the line?)


Yes, I would suggest to use the validation loss (or accuracy) to compare both approaches.