NLLB MoE router_state referenced before assignment
### System Info

- `transformers` version: 4.29.2
- Platform: Linux-5.15.0-69-generic-x86_64-with-glibc2.17
- Python version: 3.8.17
- Huggingface_hub version: 0.15.1
- Safetensors version: not installed
- PyTorch version (GPU?): 2.0.1 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

### Who can help?

@ArthurZucker @youn

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```python

model = AutoModelForSeq2SeqLM.from_pretrained("facebook/nllb-moe-54b")
model(
    input_ids=input_ids,
    attention_mask=attenstion_mask,
    decoder_input_ids=decoder_input_ids,
    decoder_attention_mask=decoder_attention_mask,
    output_router_logits=True,
    return_dict=True,
)
```

```bash
transformers/models/nllb_moe/modeling_nllb_moe.py", line 720, in forward
outputs += (router_states,)
UnboundLocalError: local variable 'router_states' referenced before assignment
```

### Expected behavior

return encoder_router_logits and decoder_router_logits rather than error. The error happens on the dense layers where no router_state is returned.
cc @ArthurZucker <|||||>Hey! Thanks for reporting! I remember working on a bug where NLLB-MoE was not being torch compiled because None values were returned. Will push a fix! 
Glad to see that Nllb-MoE is being used 🤗 