|
--- |
|
license: mit |
|
datasets: |
|
- databricks/databricks-dolly-15k |
|
language: |
|
- en |
|
pipeline_tag: text-generation |
|
tags: |
|
- dolly |
|
- dolly-v2 |
|
- instruct |
|
- sharded |
|
widget: |
|
- text: Imagine Einstein was part of a comedy duo. What would be their stage name? |
|
example_title: Einstein's comedy duo |
|
- text: What do you think Einstein's favorite Swiss chocolate brand would be? |
|
example_title: Einstein's chocolate |
|
- text: If Einstein were to enter a yodeling competition in Switzerland, what |
|
would his yodel sound like? |
|
example_title: Einstein's yodel |
|
- text: If Einstein had to create a Swiss-themed superhero, what would their name |
|
and superpower be? |
|
example_title: Swiss superhero |
|
- text: What kind of wild party would Einstein throw at ETH Zurich? |
|
example_title: Einstein's party |
|
- text: If Einstein had a pet Swiss cow, what would he name it and why? |
|
example_title: Einstein's cow |
|
- text: You've discovered a secret Swiss cheese that grants the power of genius. |
|
How would you use it to become the next Einstein? |
|
example_title: Genius cheese |
|
inference: |
|
parameters: |
|
max_length: 64 |
|
min_length: 32 |
|
--- |
|
|
|
# dolly-v2-7b: sharded checkpoint |
|
|
|
<a href="https://colab.research.google.com/gist/pszemraj/6eb7ccce28ea6aa07b8ec86388ac010e/sharded-instruction-model-playground.ipynb"> |
|
<img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/> |
|
</a> |
|
|
|
This is a sharded checkpoint (with ~4GB shards) of the `databricks/dolly-v2-7b` model. Refer to the [original model](https://huggingface.co/databricks/dolly-v2-7b) for all details. |
|
|
|
- this enables low-RAM loading, i.e. Colab :) |
|
|
|
## Basic Usage |
|
|
|
|
|
install `transformers`, `accelerate`, and `bitsandbytes`. |
|
|
|
```bash |
|
pip install -U -q transformers bitsandbytes accelerate |
|
``` |
|
|
|
Load the model in 8bit, then [run inference](https://huggingface.co/docs/transformers/generation_strategies#contrastive-search): |
|
|
|
```python |
|
from transformers import AutoTokenizer, AutoModelForCausalLM |
|
|
|
model_name = "ethzanalytics/dolly-v2-7b-sharded" |
|
tokenizer = AutoTokenizer.from_pretrained(model_name) |
|
|
|
model = AutoModelForCausalLM.from_pretrained( |
|
model_name, load_in_8bit=True, device_map="auto", |
|
) |
|
``` |
