Consumption (W)

Carbon Emitted

(tCO2eq)

Time

(GPU hours)
184320
368640
1038336
1720320
3311616

7B
13B
34B
70B

Llama 2

Total

400
400
350
400

31.22
62.44
153.90
291.42
539.00

Table 2: CO2 emissions during pretraining. Time: total GPU time required for training each model. Power
Consumption: peak power capacity per GPU device for the GPUs used adjusted for power usage efficiency.
100% of the emissions are directly offset by Meta’s sustainability program, and because we are openly releasing
these models, the pretraining costs do not need to be incurred by others.
Llama2