I am not familiar with controlling memory or memory distribution in hardware, so I cannot discuss further in the first sentence (if you have some related documents, it will help me a lot)
    // The sum of "allocated" + "free" + "cached" may be less than the
    // total capacity due to memory held by the driver and usage by other
    // programs.

Follow Pytorch github 48 (line 257-259) that I mentioned above and with your answer, I have a new question.
Is it should be:
 total = allocated + free + cached + driver # driver is 0.4 GiB, I mentioned above
       = allocated + free + (reserved - allocated) + driver # follow your answer
       = free + reserved + driver

Let take batch size = 2 to be an example, we have:
2.59 MiB + 8.37 GiB + 0.4 GiB = 8.7725 GiB 

So where is the rest of the memory? 2.2275 GiB