Update architectures/intro.md
Browse files- architectures/intro.md +7 -1
architectures/intro.md
CHANGED
|
@@ -1,2 +1,8 @@
|
|
| 1 |
Various architectures are used in code generation models, but most of them use the auto-regressive left-to-right setting, such as GPT. However InCoder used a decoder-only Transformer with Causal Masking objective,
|
| 2 |
-
that combines both next token prediction and bidirectional context through masking. AlphaCode used an encoder-decoder architecture.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
Various architectures are used in code generation models, but most of them use the auto-regressive left-to-right setting, such as GPT. However InCoder used a decoder-only Transformer with Causal Masking objective,
|
| 2 |
+
that combines both next token prediction and bidirectional context through masking. AlphaCode used an encoder-decoder architecture.
|
| 3 |
+
|
| 4 |
+
<p align="center">
|
| 5 |
+
<img src="https://huggingface.co/datasets/loubnabnl/repo-images/resolve/main/model_size.png" alt="drawing" width="440"/>
|
| 6 |
+
</p>
|
| 7 |
+
|
| 8 |
+
For model-specific information about each architecture, please select a model below:
|