Hi,
I am learning from the source code, and I got to know this line:

  
      github.com
  
  
    pytorch/pytorch/blob/8e2841781eb569a43e3f0248ba7686f9cae19b9d/aten/src/ATen/native/cuda/SoftMax.cu#L128 1

}


template<typename accscalar_t, typename Kernel>
void SpatialSoftMax_getLaunchSizes(
    Kernel k,
    uint64_t outer_size, uint64_t dim_size, uint64_t inner_size,
    dim3& grid, dim3& block, uint32_t& smem_size) {
  block = SpatialSoftMax_getBlockSize(outer_size, dim_size, inner_size);
  uint32_t block_threads = block.x * block.y;
  smem_size = block.x == 1 ? 0 : block_threads * sizeof(accscalar_t);
  int max_active_blocks;
#if defined(__HIP_PLATFORM_HCC__) && HIP_VERSION < 305
  // HIP function signature is not compatible yet.
  uint32_t max_blocks;
  cudaOccupancyMaxActiveBlocksPerMultiprocessor(&max_blocks,
                                                k, block_threads, smem_size);
  max_active_blocks = max_blocks;
#else
  cudaOccupancyMaxActiveBlocksPerMultiprocessor(&max_active_blocks,
                                                k, block_threads, smem_size);



  
  
    
    
  
  


Here we allocate as many shared memory according to the number of parallelization. However, I also learned that for most gpus, the max shm of each CUDA SM is around 48k. If there are 48 blocks, each block would have at most 1k shared memory. Would you please tell me why the pytorch code does not consider that in the implementation of operators?