<p>The diagram is a concise representation where the Nx refers to a collapsed array. This diagram can be expanded horizontally with multiple left columns chained together. This diagram can be expanded vertically with multiple yellow/orange rows chained together. If you do an image search you'll find some zoomed out color coded maps of various topologies.</p>

CC BY-SA 4.0
2023-02-11T11:18:55.493Z
Can you please provide an example of a model based on the transformer that does what you say?
nbro
CC BY-SA 4.0
2023-02-11T10:34:35.983Z
2023-02-11T10:34:35.983Z
bitRAKE
<p>I can find a million explanations of the diagram in the original transformer paper:</p>
<p><a href="https://i.sstatic.net/jIwvJ.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/jIwvJ.png" alt="enter image description here" /></a></p>
<p>But I know that modern GPT models have many millions of weights. Where are they?  Or in other words, how does this thing scale?</p>

<transformer><gpt>
How do they make transformers bigger/deeper?
Mastiff
# How do they make transformers bigger/deeper?

(Asked by: username_0 on 2023-02-10)

I can find a million explanations of the diagram in the original transformer paper:

[![enter image description here](https://i.sstatic.net/jIwvJ.png)](https://i.sstatic.net/jIwvJ.png)

But I know that modern GPT models have many millions of weights. Where are they? Or in other words, how does this thing scale?

**Tags:** transformer, gpt


---

## 1 answers

### Answer by username_1 on 2023-02-11. Score: 2

The diagram is a concise representation where the Nx refers to a collapsed array. This diagram can be expanded horizontally with multiple left columns chained together. This diagram can be expanded vertically with multiple yellow/orange rows chained together. If you do an image search you'll find some zoomed out color coded maps of various topologies.

> **Comments:**
> **username_2**: Can you please provide an example of a model based on the transformer that does what you say?  

---

