readme: ---
  - Text Generation
  - PyTorch
  - Transformers
  - gpt_neo
  - text generation
  ---
  
  ## Petrained Model Description: Open Source Version of GPT-3
  Generative Pre-trained Transformer 3 (GPT-3) is an autoregressive language model that uses deep learning to produce human-like text.
  It is the third-generation language prediction model in the GPT-n series (and the successor to GPT-2) created by OpenAI
  
  GPT-Neo (125M) is a transformer model designed using EleutherAI's replication of the GPT-3 architecture. GPT-Neo refers to the class of models, while 125M represents the number of parameters of this particular pre-trained model.
  and first released in this [repository](https://github.com/EleutherAI/gpt-neo). 
  
  
  ## Fine-tuned Model Description: GPT-3 fine-tuned Multi-XScience
  The Open Source version of GPT-3: GPT-Neo(125M) has been fine-tuned on a dataset called "Multi-XScience": [Multi-XScience_Repository](https://github.com/yaolu/Multi-XScience): A Large-scale Dataset for Extreme Multi-document Summarization of Scientific Articles.    
  
  I first fine-tuned and then deployed it using Google "Material Design" (on Anvil): [Abir Scientific text Generator](https://abir-scientific-text-generator.anvil.app/)  
  
  By fine-tuning GPT-Neo(Open Source version of GPT-3), on Multi-XScience dataset, the model is now able to generate scientific texts(even better than GPT-J(6B).   
  Try putting the prompt "attention is all" on both my [Abir Scientific text Generator](https://abir-scientific-text-generator.anvil.app/)  and on the [ GPT-J Eleuther.ai Demo](https://6b.eleuther.ai/) to understand what I mean.   
  And Here's a demonstration video for this. [Video real-time Demontration](https://www.youtube.com/watch?v=XP8uZfnCYQI)