Thanks @albanD!
The issue is bit weird, It is actually working fine If I simply replace the ‘Infinity’ to just calculate the l2 norm, without any  issue. Also I reduced the batch size to as low as 4. But still the same error persists. Do you think this could be due to some other reason?