Thanks a lot, Bhack with your help I was able to fix it.
It seems during the conversion of the generator to the dataset object length of the dataset is unknown and infinite. By using the tf.data.experimental.cardinality() we can get the number of samples in our dataset.
Like as I said before during the conversion the length is infinite and unknown so it will return -2 .
We can fix this by entering the number of samples explicitly in our dataset by using tf.data.experimental.assert_cardinality(num_of_samples) and now we can even use the len() function.
I have shared the link to the complete notebook here Google Colaboratory 30
Best,
Ashik