readme: ---
  language: 
    - en
  datasets:
  - The Pile
  - super_glue
  tags:
  - pytorch
  - causal-lm
  - 8bit
  inference: false
  ---
  
  side project, not final working good product
  
  see [this repo](https://huggingface.co/crumb/gpt-j-6b-shakespeare) for more information, finetuned with [8-bit Adam](https://arxiv.org/abs/2110.02861) on custom transformed super glue tasks, transformed somewhat like the [T0 paper](https://arxiv.org/abs/2110.08207)
  
  ```python
  from IPython import display 
  !pip install transformers==4.14.1 -q
  !pip install bitsandbytes-cuda111==0.26.0 -q
  !pip install git+https://github.com/aicrumb/transformers-8bit -q
  
  import transformers_8bit
  
  model, tokenizer, config = transformers_8bit.gptj("crumb/gpt-j-6b-finetune-super-glue", device='cuda')
  
  prompt = tokenizer("<QUERY> If birds are in group B, and snakes are in group A, what group are pythons in? <RESPONSE>", return_tensors='pt')
  prompt = {key: value.to('cuda') for key, value in prompt.items()}
  length = len(prompt['input_ids'][0])
  out = model.generate(**prompt, min_length=length+2, max_length=length+2, do_sample=False, pad_token_id=tokenizer.eos_token_id)
  print(tokenizer.decode(out[0]))
  
  """output
  <QUERY> If birds are in group B, and snakes are in group A, what group are pythons in? <RESPONSE> Group A
  """
  ```