The argmin operation is non-differentiable (it returns an integer number). But the operation that returns the min value is. The gradient is just 1 for the value that was selected and 0 for all the others.