Answers.Body: <p>Depending on how much you want to understand about transformers, it always helps knowing the math first and to really understand how transformers work. So, to answer your question, what is a place to start reading:</p>
  <p>As a good start I would suggest you read the paper &quot;Attention is all you need&quot; alongside the <a href="https://nlp.seas.harvard.edu/2018/04/03/attention.html" rel="nofollow noreferrer">Annotated Transformer</a> tutorial, which is a good “annotated” version of the paper in the form of a line-by-line implementation. Furthermore Andrej Karpathy provides a very helpful and understandable tutorial on LLMs and on how to build GPT from scratch, check out <a href="https://karpathy.ai/" rel="nofollow noreferrer">his website</a> for more information.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-02-07T10:12:47.160Z
Answers.LastActivityDate: 2024-02-07T10:12:47.160Z
Answers.UserDisplayName: tanasr
Body: <p>As I'm completely new to this field, I find it hard to get started given the requirements I have. I'm a bit overwhelmed by all the models and options that are available. Even though it wasn't difficult to run an LLM locally:</p>
  <pre><code><span class="math-container">$&gt; brew install llm
  $</span>&gt; llm -m gpt4all-13b-snoozy-q4_0  &quot;Tell me a joke&quot;
  </code></pre>
  <p>despite being very nice, it is not what I'm looking for. I want to train a model with my own data. Also, I want to build a website around it, so it means I can access the LLM with APIs.</p>
  <p>So, as you can see I have some requirements. What would be a great place to start reading? A good tutorial to get started would be very much appreciated!</p>
  
Tags: <training><transformer><large-language-models><gpt>
Title: Getting started with training local LLM using python
UserDisplayName: Jeanluca Scaljeri
ThreadText: # Getting started with training local LLM using python
  
  (Asked by: username_0 on 2024-02-06)
  
  As I'm completely new to this field, I find it hard to get started given the requirements I have. I'm a bit overwhelmed by all the models and options that are available. Even though it wasn't difficult to run an LLM locally:
  
  ```
  $> brew install llm
  $> llm -m gpt4all-13b-snoozy-q4_0  "Tell me a joke"
  
  ```
  
  despite being very nice, it is not what I'm looking for. I want to train a model with my own data. Also, I want to build a website around it, so it means I can access the LLM with APIs.
  
  So, as you can see I have some requirements. What would be a great place to start reading? A good tutorial to get started would be very much appreciated!
  
  **Tags:** training, transformer, large-language-models, gpt
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2024-02-07. Score: 2
  
  Depending on how much you want to understand about transformers, it always helps knowing the math first and to really understand how transformers work. So, to answer your question, what is a place to start reading:
  
  As a good start I would suggest you read the paper "Attention is all you need" alongside the [Annotated Transformer](https://nlp.seas.harvard.edu/2018/04/03/attention.html) tutorial, which is a good “annotated” version of the paper in the form of a line-by-line implementation. Furthermore Andrej Karpathy provides a very helpful and understandable tutorial on LLMs and on how to build GPT from scratch, check out [his website](https://karpathy.ai/) for more information.
  
  
  ---
  
  