 Model Configuration  | Model         | n_params | n_enc-layers | n_dec-layers | d_model | d_ff  | n_heads | d_head | attn-scale | | ------------- | -------- | ------------ | ------------ | ------- | ----- | ------- | ------ | ---------- | | EVA1.0        | 2.8B     | 24           | 24           | 2,048   | 5,120 | 32      | 64     | No         | | EVA2.0_Base   | 300M     | 12           | 12           | 768     | 3,072 | 12      | 64     | Yes        | | EVA2.0_Large  | 970M     | 24           | 24           | 1,024   | 4,096 | 16      | 64     | Yes        | | EVA2.0_xLarge | 2.8B     | 24           | 24           | 2,048   | 5,120 | 32      | 64     | Yes        |  