Thank you @ptrblck for your reply.
After getting a response from Fairseq team they suggested me to use the preprocessing module because --raw text and memory indexing code couldn’t scale up for larger dataset. I have used the preprocessing module of fairseq and it seems working now.
Thank you for helping me