I have a similar, maybe the same problem.
An image_hw with values between 0.0 and 1.0, where the non-zero pixels are somewhere in the center (i.e. unknown padding on left, right, top, and bottom, maybe no padding).
An example image:

I need a bounding box coordinates so that is can rectangle crop center region with non-zero pixels. image_hw[top:bottom, left:right]
I can do this with 4 for loops, similar to:
for top_pad in range(image_hw.shape[0];
   if image_hw[top_pad,:].sum()>0:
      break;

I suspect pytorch and a function that could accomplish this more concisely.