content.raw: 🔮 GPT-3 implemented in pure Free Pascal!
content.type: text
content.value: 🔮 GPT-3 implemented in pure Free Pascal!
content.type: new_line
content.href: https://github.com/joaopauloschuler/gpt-3-for-pascal
content.raw: https://github.com/joaopauloschuler/gpt-3-for-pascal
content.type: link
content.type: new_line
content.type: new_line
content.raw: This implementation follows the GPT-3 Small architecture from the landmark paper "Language Models are Few-Shot Learners":
content.type: text
content.value: This implementation follows the GPT-3 Small architecture from the landmark paper "Language Models are Few-Shot Learners":
content.type: new_line
content.code: ┌─────────────────────────┐
  │     Input Layer       │
  ├─────────────────────────┤
  │ Token & Positional    │
  │     Embedding         │
  ├─────────────────────────┤
  │   12x Transformer     │
  │      Blocks           │
  │  - 12 heads           │
  │  - 768 hidden dims    │
  │  - 3072 intermediate  │
  ├─────────────────────────┤
  │   Output Layer        │
  └─────────────────────────┘
content.raw: ```
  ┌─────────────────────────┐
  │     Input Layer       │
  ├─────────────────────────┤
  │ Token & Positional    │
  │     Embedding         │
  ├─────────────────────────┤
  │   12x Transformer     │
  │      Blocks           │
  │  - 12 heads           │
  │  - 768 hidden dims    │
  │  - 3072 intermediate  │
  ├─────────────────────────┤
  │   Output Layer        │
  └─────────────────────────┘
  ```
content.type: code_fence
content.type: new_line
content.raw: Clean Pascal Implementation
content.type: text
content.value: Clean Pascal Implementation
content.type: new_line
content.code: for CntLayer := 1 to {Layers=}12 do
  begin
    Result.AddTransformerBlockCAI(
      {Heads=}12, 
      {intermediate dimensions=}4*768, 
      {NoForward=}true, 
      {HasNorm=}true, 
      false
    );
  end;
content.raw: ```
  for CntLayer := 1 to {Layers=}12 do
  begin
    Result.AddTransformerBlockCAI(
      {Heads=}12, 
      {intermediate dimensions=}4*768, 
      {NoForward=}true, 
      {HasNorm=}true, 
      false
    );
  end;
  ```
content.type: code_fence
content.type: new_line
rawContent: 🔮 GPT-3 implemented in pure Free Pascal!
  https://github.com/joaopauloschuler/gpt-3-for-pascal
  
  This implementation follows the GPT-3 Small architecture from the landmark paper "Language Models are Few-Shot Learners":
  ```
  ┌─────────────────────────┐
  │     Input Layer       │
  ├─────────────────────────┤
  │ Token & Positional    │
  │     Embedding         │
  ├─────────────────────────┤
  │   12x Transformer     │
  │      Blocks           │
  │  - 12 heads           │
  │  - 768 hidden dims    │
  │  - 3072 intermediate  │
  ├─────────────────────────┤
  │   Output Layer        │
  └─────────────────────────┘
  ```
  Clean Pascal Implementation
  ```
  for CntLayer := 1 to {Layers=}12 do
  begin
    Result.AddTransformerBlockCAI(
      {Heads=}12, 
      {intermediate dimensions=}4*768, 
      {NoForward=}true, 
      {HasNorm=}true, 
      false
    );
  end;
  ```
  
reactions.reaction: 😎
reactions.users: John6666 darkzbaron victor Boqua benhaotang yattishr dantezxcd
reactions.reaction: 👍
reactions.users: krinal dantezxcd
reactions.reaction: ❤️
reactions.users: DonkeySmall dantezxcd