This is not a perfect solution, but at one point I went through most of the code for Faster RCNN  from pytorch and added comments to help me figure out what was going on. You can seen them on github: https://github.com/DerekGloudemans/torchvision-detection-clone 4. I’d also recommend reading the NIPS paper for a better understanding of what’s going on. In general, if you’re interested in learning about object detection as a whole there are many resources, but if you are interested in learning the finer working details of FasterRCNN diving into the code is probably the way to go.