Hi, @KhusainovAidar
I found one solution to convert fairseq pre-trained model to Transformers format. For your reference.
python -m  transformers.models.wav2vec2.convert_wav2vec2_original_pytorch_checkpoint_to_pytorch --pytorch_dump_folder_path ./converted_model/ --checkpoint_path /path/to/**.pt  --not_finetuned