GPT 3
[In-context learning](https://texonom.com/in-context-learning-2f55f51a87a04785be78dc65ef752ca4) [RLHF](https://texonom.com/rlhf-4b184f9c9e8b4c7a8861fb6374e91aa6) 


175B, 600GB data

[Text DaVinci 003](https://texonom.com/text-davinci-003-fba990de22c745a9ab9298657a7e4b94)


![https://yaofu.notion.site/How-does-GPT-Obtain-its-Ability-Tracing-Emergent-Abilities-of-Language-Models-to-their-Sources-b9a57ac0fcf74f30a1ab9e3e36fa1dc1](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2F9bd0bef7-8e88-4a1e-b409-3e6dc6cfbf66%2FUntitled.png?table=block&id=018dd5c0-9d94-47d0-b5a3-3bcd61dceaad&cache=v2)


### Why gpt fails on complex task
> [openai-cookbook/techniques_to_improve_reliability.md at main · openai/openai-cookbook](https://github.com/openai/openai-cookbook/blob/main/techniques_to_improve_reliability.md)

### natural language to code
> [Microsoft uses GPT-3 to let you code in natural language](https://techcrunch.com/2021/05/25/microsoft-uses-gpt-3-to-let-you-code-in-natural-language/?utm_source=tldrnewsletter)

> [A robot wrote this entire article. Are you scared yet, human? | GPT-3](https://www.theguardian.com/commentisfree/2020/sep/08/robot-wrote-this-article-gpt-3)

> [OpenAI begins allowing customers to fine-tune GPT-3](https://venturebeat.com/2021/12/14/openai-begins-allowing-customers-to-fine-tune-gpt-3/)

> [https://simonwillison.net/2022/Jul/9/gpt-3-explain-code/](https://simonwillison.net/2022/Jul/9/gpt-3-explain-code/)

> [The GPT-3 Architecture, on a Napkin](https://dugas.ch/artificial_curiosity/GPT_architecture.html)




