Answers.Body: <blockquote>
  <p>Are the model implementations in Hugging Face’s transformers library created by the original model authors or by Hugging Face?</p>
  </blockquote>
  <p>Depends on the model.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-10-07T05:04:04.320Z
Answers.LastActivityDate: 2024-10-07T05:04:04.320Z
Answers.UserDisplayName: Franck Dernoncourt
Answers.Body: <p>You can actually see the author of this code if you look at the first few lines of the file.</p>
  <p>To save you clicking the link again:</p>
  <pre><code># coding=utf-8
  # Copyright 2022 EleutherAI and the HuggingFace Inc. team. All rights reserved.
  #
  # This code is based on EleutherAI's GPT-NeoX library and the GPT-NeoX
  # and OPT implementations in this library. It has been modified from its
  # original forms to accommodate minor architectural differences compared
  # to GPT-NeoX and OPT used by the Meta AI team that trained the model.
  #
  </code></pre>
  <p>So this file is <strong>not</strong> written by meta.</p>
  <p>You can find code written by the meta team that implements llama inference here: <a href="https://github.com/Meta-Llama/llama" rel="noreferrer">https://github.com/Meta-Llama/llama</a> and <a href="https://github.com/meta-llama/llama/blob/main/llama/model.py" rel="noreferrer">https://github.com/meta-llama/llama/blob/main/llama/model.py</a></p>
  <p>The main difference between these two implementations is that the EleutherAI/Huggingface implementation is built on top of the <code>transformers</code> API (which I will point out does use <code>pytorch</code>), whereas the meta implementation is written just on top of <code>pytorch</code>. If you are familiar with the <code>transformers</code> API, the EleutherAI/Huggingface implementation may be a little bit more convenient to use.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-09-26T08:43:56.597Z
Answers.LastActivityDate: 2024-09-26T08:48:16.077Z
Answers.LastEditDate: 2024-09-26T08:48:16.077Z
Answers.UserDisplayName: Aleph 0
Body: <p>I've been exploring the implementation of models like Llama in Hugging Face’s <code>transformers</code> library, for example:<br />
  <a href="https://github.com/huggingface/transformers/blob/main/src/transformers/models/llama/modeling_llama.py" rel="noreferrer">Hugging Face's Llama model implementation</a>.</p>
  <p>I’m curious about how these implementations work:</p>
  <ul>
  <li>Are the model codes in Hugging Face’s <code>transformers</code> library (e.g., for Llama) written directly by the creators of the models, or is it Hugging Face’s attempt to reproduce the functionality based on documentation or papers released by the model's original creators?</li>
  <li>How closely do these implementations follow the specifications or official codebases (if available) provided by the model authors?</li>
  </ul>
  
Tags: <machine-learning><natural-language-processing><transformer><large-language-models><huggingface>
Title: Are the model implementations in Hugging Face’s transformers library created by the original model authors or by Hugging Face?
UserDisplayName: mlibre
ThreadText: # Are the model implementations in Hugging Face’s transformers library created by the original model authors or by Hugging Face?
  
  (Asked by: username_0 on 2024-09-26)
  
  I've been exploring the implementation of models like Llama in Hugging Face’s `transformers` library, for example:  
  [Hugging Face's Llama model implementation](https://github.com/huggingface/transformers/blob/main/src/transformers/models/llama/modeling_llama.py).
  
  I’m curious about how these implementations work:
  
  * Are the model codes in Hugging Face’s `transformers` library (e.g., for Llama) written directly by the creators of the models, or is it Hugging Face’s attempt to reproduce the functionality based on documentation or papers released by the model's original creators?
  * How closely do these implementations follow the specifications or official codebases (if available) provided by the model authors?
  
  **Tags:** machine-learning, natural-language-processing, transformer, large-language-models, huggingface
  
  ---
  
  ## 2 answers
  
  ### Answer by username_2 on 2024-09-26. Score: 5 | Accepted answer
  
  You can actually see the author of this code if you look at the first few lines of the file.
  To save you clicking the link again:
  # coding=utf-8
  # Copyright 2022 EleutherAI and the HuggingFace Inc. team. All rights reserved.
  #
  # This code is based on EleutherAI's GPT-NeoX library and the GPT-NeoX
  # and OPT implementations in this library. It has been modified from its
  # original forms to accommodate minor architectural differences compared
  # to GPT-NeoX and OPT used by the Meta AI team that trained the model.
  #
  
  So this file is not written by meta.
  You can find code written by the meta team that implements llama inference here: https://github.com/Meta-Llama/llama and https://github.com/meta-llama/llama/blob/main/llama/model.py
  The main difference between these two implementations is that the EleutherAI/Huggingface implementation is built on top of the transformers API (which I will point out does use pytorch), whereas the meta implementation is written just on top of pytorch. If you are familiar with the transformers API, the EleutherAI/Huggingface implementation may be a little bit more convenient to use.
  
  
  ---
  
  ### Answer by username_1 on 2024-10-07. Score: 1
  
  > Are the model implementations in Hugging Face’s transformers library created by the original model authors or by Hugging Face?
  
  Depends on the model.
  
  
  ---
  
  