Small bug while converting wav2vec2 model trained using fairseq to huggingface
Hi,

I was trying to convert a wav2vec2 model trained using fairseq to have support with HuggingFace but there is a small error in inference.

When I use the code below:


```
import soundfile as sf
import torch
from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor

# load pretrained model
processor = Wav2Vec2Processor.from_pretrained('hf/output')
model = Wav2Vec2ForCTC.from_pretrained("hf/output")


# load audio
audio_input, sample_rate = sf.read('004-M-23_001.wav')

# pad input values and return pt tensor
input_values = processor(audio_input, sampling_rate=sample_rate, return_tensors="pt").input_values

# INFERENCE

# retrieve logits & take argmax
logits = model(input_values).logits
predicted_ids = torch.argmax(logits, dim=-1)

# transcribe
transcription = processor.decode(predicted_ids[0])

```

I get this as the output:

```
<s>ह<s>ॉ<s>ट<s>ल<s> <s>र<s>ॉ<s>य<s>ल<s> <s>ह<s>े<s>र<s>ि<s>ट<s>े<s>ज<s> क<s>े<s> <s>च<s>ी<s>ज<s> <s>क<s>े<s> <s>ए<s>क<s> <s>ब<s>ह<s>ु<s>त<s> <s>अ<s>च्<s>छ<s>ा<s> <s>ह<s>ै<s> <s>क<s>्य<s>ा<s> <s>
```

but, I should ideally get this:

```
हॉटल रॉयल हेरिटेज के चीज के एक बहुत अच्छा है क्या
```

I can easily solve this by using:

```
print(transcription.replace('<s>', ''))
```

But if I deploy the model, the output of inference will contain ```<s>``` as I cannot change the output of deployed model.

Can you please let me know if I am doing any mistake in the conversion process.

My vocab.json looks like this:

```
{"<s>": 0, "<pad>": 1, "</s>": 2, "<unk>": 3, "|": 4, "0": 5, "1": 6, "2": 7, "3": 8, "4": 9, "5": 10, "6": 11, "7": 12, "8": 13, "9": 14, "ँ": 15, "ं": 16, "ः": 17, "अ": 18, "आ": 19, "इ": 20, "ई": 21, "उ": 22, "ऊ": 23, "ऋ": 24, "ए": 25, "ऐ": 26, "ऑ": 27, "ओ": 28, "औ": 29, "क": 30, "ख": 31, "ग": 32, "घ": 33, "ङ": 34, "च": 35, "छ": 36, "ज": 37, "झ": 38, "ञ": 39, "ट": 40, "ठ": 41, "ड": 42, "ढ": 43, "ण": 44, "त": 45, "थ": 46, "द": 47, "ध": 48, "न": 49, "प": 50, "फ": 51, "ब": 52, "भ": 53, "म": 54, "य": 55, "र": 56, "ल": 57, "व": 58, "श": 59, "ष": 60, "स": 61, "ह": 62, "ा": 63, "ि": 64, "ी": 65, "ु": 66, "ू": 67, "ृ": 68, "ॅ": 69, "े": 70, "ै": 71, "ॉ": 72, "ो": 73, "ौ": 74, "्": 75, "क़": 76, "ख़": 77, "ग़": 78, "ज़": 79, "ड़": 80, "ढ़": 81, "फ़": 82, "य़": 83}
```
Hi @harveenchadha 

if you pass `skip_special_tokens=True` to `decode` method, it will skip all the special tokens. <|||||>Hi Suraj,

Thanks! That works. But in the inference service that is deployed to test in browser, how will this change?<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.