readme: ---
  duplicated_from: Rallio67/chip_12B_instruct_alpha
  ---
  
  ## FORK OF Rallio67/chip_12B_instruct_alpha
  
  Experimental 12B instruction tuned model based on pythia-12B-deduped
  --------------------------------------------------
  # Model may create undesirable content, use at your own risk. 
  Finetuned on a variety of instruction datasets. 
  See: https://github.com/Rallio67/language-model-agents 
  
  # Thanks to LAION contributors and Stability.ai
  for help building datasets and compute resources.
  
  # Prompt the  model by typing:
  User: followed by your question. The agent will reply as Chip \
  Chip is loosely inspired by the fictional character Chip the robot. \
  see: https://en.wikipedia.org/wiki/Not_Quite_Human_(film))
  
  # For multiple rounds of dialogue:
  After your question add two new lines followed by Chip:
  (\n\nChip:)
  
  # Example code snippet to run on your own system:
  ```
  from transformers import AutoModelForCausalLM, AutoTokenizer
  import torch
  
  chip_map= {'gpt_neox.embed_in': 0,
   'gpt_neox.layers': 0,
   'gpt_neox.final_layer_norm': 0,
   'embed_out': 0}
  
  name = "Rallio67/chip_12B_instruct_alpha"
  model = AutoModelForCausalLM.from_pretrained(name, device_map=chip_map, torch_dtype=torch.float16, load_in_8bit=True )
  tokenizer = AutoTokenizer.from_pretrained(name)
  
  def generate_from_model(model, tokenizer):
      encoded_input = tokenizer(text, return_tensors='pt')
      output_sequences = model.generate(
                                      input_ids=encoded_input['input_ids'].cuda(0),
                                      do_sample=True,
                                      max_new_tokens=35,
                                      num_return_sequences=1,
                                      top_p=0.95,
                                      temperature=0.5,
                                      penalty_alpha=0.6,
                                      top_k=4,
                                      output_scores=True,
                                      return_dict_in_generate=True,
                                      repetition_penalty=1.03,
                                      eos_token_id=0,
                                      use_cache=True
                                    )
      gen_sequences = output_sequences.sequences[:, encoded_input['input_ids'].shape[-1]:]
      for sequence in gen_sequences:
          new_line=tokenizer.decode(sequence, skip_special_tokens=True)
          print(new_line)
  
  text = "User: Why is everyone so excited about AI chatbots these days?"
  generate_from_model(model,tokenizer)
  #Chip: Chatbots are a great way to automate simple tasks. They can help you save time and make your life easier. For example, they can
  ```
  
  Sampling Settings:
  -------
  top_p=0.95,
  temperature=0.5,
  penalty_alpha=0.6,
  top_k=4,
  repetition_penalty=1.03
  
  Example Generations:
  ---------------
  >User: Are a manatee and a dolphin more likely to be friends or foes?
  
  
  Chip: A manatee and a dolphin are both mammals, so they are more likely to be friends than foes.
  
  
  Chip: I’m not sure. I’d have to look into it.
  
  
  Chip: I’d say they’re more likely to be enemies. A manatee is a mammal, and a dolphin is a mammal, so they’re more likely to be enemies than friends.
  
  
  Chip: A manatee and a dolphin are more likely to be friends. Manatees and dolphins are both marine mammals that live in the ocean. They have similar habitats and lifestyles, and they are both herbivores. They also have similar diets, so they can eat the same food.
  
  >User: Why are roses red?
  
  Chip: Roses are red because they are made up of a pigment called anthocyanin. Anthocyanin is a pigment that gives the plant its color. It is also found in other flowers, such as daisies, tulips, and violets.
  
  Chip: Roses are red because they contain a pigment called anthocyanin. Anthocyanins are a type of flavonoid that gives plants their color. Flavonoids are a class of compounds found in plants and other organisms.
  
  Chip: Roses are red because they are a member of the rose family. The rose family is a large group of plants that includes many different species. The most common type of rose is the garden rose, which is a hybrid of two species of roses. The two species are Rosa gallica and Rosa rugosa. The rose family is named after the genus name of the first species, Rosa.
  
  Chip: Roses are red because they have a gene that makes them red. The gene is called the anthocyanin gene, and it is found in most plants. It is also found in some other animals, such as birds, lizards, and insects. The anthocyanin gene is responsible for the color of the flower.
  
  >User: What's a good recipe for bbq ribs?
  
  Chip: Ingredients:
   • 4 racks of baby back ribs
   • 1 bottle of your favorite BBQ sauce
   • 1/2 cup of brown sugar
   • 1/2 cup of honey
   • 1/4 cup of apple cider vinegar
   • 2 cloves of garlic, minced
   • 2 tablespoons of chili powder
   • 1 teaspoon of paprika
   • 1/2 teaspoon of cayenne pepper
   • 1/4 teaspoon of black pepper
   • 1/4 teaspoon of salt
  
  Instructions:
  1. Preheat oven to 350 degrees F (175 degrees C).
  2. Place the ribs in 