We had a bug where _th_get_device was taking much more time than it should have been, as you’ve encountered here. I’ve fixed this on master (see https://github.com/pytorch/pytorch/issues/13049 13); you should try a nightly build and see if performance gets better for you.
The best way to look up what the various function calls are doing are to grep the codebase. If you have specific questions about where to find code pointers for specific questions, feel free to ask.