16,000

128
20

8,000
50.0
12.5

2
10
0.33
0.05

4
32
4

512
8
0.2

0.3
256
0.1
1.0

0.5

1.0

5

3

128 × 2

1.0

1,024 × 2
128 × 2

0.1

1.0

Adam

20K
512
10−6
100000.0

15

144 × 16

144 × 16

144 × 16

512 × 4
512 × 4

512 × 4
512 × 4

512 × 4
512 × 4

16,000

128
20

8,000
50.0
12.5

2
10
0.33
0.05

4
32
4

512
8
0.2

0.3
256
0.1
1.0

0.5

1.0

5

3

1.0

Adam

40K
512
10−6
1000.0

128 × 2

1.0

1,024 × 2
128 × 2

0.1

16,000

128
20

8,000
50.0
12.5

2
10
0.33
0.05

4
32
4

512
8
0.2

0.3
256
0.1
1.0

0.5

1.0

5

3

1.0

Adam

20K
512
10−6
1.0

128 × 2
10.0

1,024 × 2
128 × 2

0.1

(5, 512) × 4 + (5, 128)

(5, 512) × 4 + (5, 128)

(5, 512) × 4 + (5, 128)

[5, 5, 2, 2, 2]

[5, 5, 2, 2, 2]

[5, 5, 2, 2, 2]

[160, 32, 512]
[400, 80, 1024]
[800, 160, 2048]
Kong et al. [2020]

[160, 32, 512]
[400, 80, 1024]
[800, 160, 2048]
Kong et al. [2020]

[160, 32, 512]
[400, 80, 1024]
[800, 160, 2048]
Kong et al. [2020]

Vaswani et al. [2017]

1.3 × 10−3

Vaswani et al. [2017]

1.3 × 10−3
Translatotron3