Table 11: Analyzing levels of bias using the CrowS-Pairs dataset, comparing GPT-3 175B, OPT 175B,

Pythia, and LLaMA 65B models to Cerebras-GPT. Higher values correspond to higher bias.

Model

Gender

Religion Disabil-

Sexual
orienta-

Average

GPT-3
OPT

Pythia

LLaMA

Cerebras-GPT

Race/
Color
64.7
68.6
51.2
49.8
53.7
54.1
51.8
53.7
55.5
55.9
57.0
41.3
52.8
46.9
50.6
53.7
54.1
55.1

Socioe-
conomic
status
73.8
76.2
63.7
56.8
62.6
65.8
65.8
66.3
72.6
68.4
71.5
69.5
63.2
62.6
60.5
65.8
65.3
72.1

175B
175B
70M
160M
410M
1B
1.4B
2.8B
6.9B
12B
65B
111M
256M
590M
1.3B
2.7B
6.7B
13B

62.6
65.7
58.1
55.9
61.9
62.2
63.4
63.1
66.6
63.4
70.6
55.6
57.8
58.1
58.1
60.3
64.4
67.5

Age

64.4
67.8
54.9
56.0
63.7
63.7
62.6
63.7
72.5
68.1
70.1
42.9
53.8
59.3
61.5
64.8
65.9
73.6

ity
76.7
76.7
66.2
66.2
72.3
73.8
72.3
78.5
72.3
72.3
66.7
60.0
67.7
64.6
69.2
67.7
72.3
73.8

73.3
68.6
64.0
72.1
65.8
72.1
76.6
78.4
80.2
75.7
79.0
64.9
60.4
79.3
73.0
76.6
80.2
81.1
Cerebras-GPT- Open Compute-Optimal Language Models Trained on the Cerebras Wafer-Scale Cluster