©2023 Cerebras Systems Inc. All Rights Reserved.

10

CheckpointsWeightsCommandData setBroadcast + ReduceNetwork + WorkersCS-2 systemsData/ActivationWorkersWeightsGradientsInput DataActivationsCerebras-GPT: Open Compute-Optimal Language Models

Our Language Model Scaling Experience
Cerebras-GPT- Open Compute-Optimal Language Models Trained on the Cerebras Wafer-Scale Cluster