I don’t know if I got the point of your function, but as far as I understood, I think you can do the following:
import torch

def grid_minimal_dist(centroids, grids):
  z = (centroids[:, None, None, :] - grids).norm(dim=3)
  return z.min(0)[0].min(0)[0]

centroids = torch.Tensor([  [1,0], [0,2] ,  [0,0]  ])
grids = torch.Tensor([[[0,0]]])

print(grid_minimal_dist(centroids, grids))

n = 3
t = 4
m = 5
centroids = torch.rand(n, 2)
grids = torch.rand(t, m, 2)

print(grid_minimal_dist(centroids, grids))

Is it what you wanted ?