Hi all,
I am a beginner of pytorch, and I am trying to implement a complex CNN model called FEC-CNN from paper “A Fully End-to-End Cascaded CNN for Facial Landmark Detection 88”. However, I met some problem while building it.
Here is the architecture of FEC-CNN:

22312055_1605985996090015_1530419928_o.jpg1167×607 104 KB

And here is the architecture of a single sub-CNN:

22330951_1605985989423349_343220074_n.jpg428×518 56.1 KB

Explaining the model a bit:
The input of FEC-CNN model is face images, and the output is 68 landmarks of those images.
First, an initial CNN model will predict the initial 68 landmarks of the image, and those landmark will be refined by the following sub-CNNs each of which takes patches around landmarks predicted by previous stage as input.
There are my questions:
1, How to crop image tensor based on the output from the CNN model? I know that it is very convenient for pytorch to convert tensor into numpy, but is it correct for me to convert tensor into numpy for such processing and then convert it back to tensor for the next model input? I want the gradients can be computed from the start to the very end.
2, Is it possible for me to set a batch size, since I have to crop every image based on different landmarks predicted by models? It seems to me that models will handle inputs batch size by batch size, so I have no idea how to crop every image with different cropping positions per batch size but using batch size = 1.
Sorry for not giving any codes, I have been stuck with these problems for a week.
Thank you for any answer, or possible hint.