You should be able to use these structs that are autogenerated and should be included in the main function you write in in the C++ that drives inference:

  

      github.com
  

  
    pytorch/glow/blob/b0de69a7639567bfec9081ab6a8bcb792638cba9/lib/LLVMIRCodeGen/BundleSaver.cpp#L106-L135 3



      
      
          /// Header file common definitions for dynamic API.
          static const char *dynamicApiCommonDefines = R"RAW(
          // Type describing a symbol table entry of a generated bundle.
          struct SymbolTableEntry {
            // Name of a variable.
            const char *name;
            // Offset of the variable inside the memory area.
            uint64_t offset;
            // The number of elements inside this variable.
            uint64_t size;
            // Variable kind: 1 if it is a mutable variable, 0 otherwise.
            char kind;
          };
          
          
// Type describing the config of a generated bundle.
          struct BundleConfig {
            // Size of the constant weight variables memory area.
            uint64_t constantWeightVarsMemSize;
            // Size of the mutable weight variables memory area.
            uint64_t mutableWeightVarsMemSize;
      
    
  

  This file has been truncated. show original

  

  
    
    
  

  


E.g. using the BundleConfig you can understand the total memory usage of the network itself, such as the size of constant weights, etc. Aside from that if you need more info about data transfer then yes you would need to take a look at the actual data you’re passing into the network in the C++ file that drives inference.