The overhead here isn’t from any computation (there isn’t much computation going on in the example) but from the bookkeeping, i.e. the creation of all the Tensors - memory allocation setting up tensors etc., so this is expected.
In the example even the - considerable smaller - overhead of vector allocations is not accounted for, things are done in-place to avoid allocations etc., so it is much of an apples-to-oranges comparison.
Once you have a non-negligble amount of computation, you find that this overhead is dwarfed by what’s going on in the actual backend routines.
I don’t think that PyTorch needs to hide in terms of speed but there are a few where we’d like things to be faster, see this great detailed analysis of overheads in LSTMs 76 by @richard for a taste of this.
Best regards
Thomas