where we set the Power Usage Effectiveness (PUE)
at 1.1. The resulting carbon emission depends on
the location of the data center used to train the net-
work. For instance, BLOOM uses a grid that emits
0.057 kg CO2eq/KWh leading to 27 tCO2eq and
OPT a grid that emits 0.231 kg CO2eq/KWh, lead-
ing to 82 tCO2eq. In this study, we are interested in
comparing the cost in carbon emission of training
of these models if they were trained in the same
data center. Hence, we do not take the location
of data center in consideration, and use, instead,
the US national average carbon intensity factor of
0.385 kg CO2eq/KWh. This leads to the following
formula for the tons of carbon emissions:

tCO2eq = MWh × 0.385.
LLaMA- Open and Efficient Foundation Language Models