|
--- |
|
license: cc-by-nc-sa-4.0 |
|
language: |
|
- en |
|
library_name: transformers |
|
pipeline_tag: text-generation |
|
tags: |
|
- stableLM |
|
- sharded |
|
widget: |
|
- text: Imagine Einstein was part of a comedy duo. What would be their stage name? |
|
example_title: Einstein's comedy duo |
|
- text: What do you think Einstein's favorite Swiss chocolate brand would be? |
|
example_title: Einstein's chocolate |
|
- text: If Einstein were to enter a yodeling competition in Switzerland, what |
|
would his yodel sound like? |
|
example_title: Einstein's yodel |
|
- text: If Einstein had to create a Swiss-themed superhero, what would their name |
|
and superpower be? |
|
example_title: Swiss superhero |
|
- text: What kind of wild party would Einstein throw at ETH Zurich? |
|
example_title: Einstein's party |
|
- text: If Einstein had a pet Swiss cow, what would he name it and why? |
|
example_title: Einstein's cow |
|
- text: You've discovered a secret Swiss cheese that grants the power of genius. |
|
How would you use it to become the next Einstein? |
|
example_title: Genius cheese |
|
inference: |
|
parameters: |
|
max_length: 96 |
|
min_length: 32 |
|
--- |
|
# StableLM-Tuned-Alpha: sharded checkpoint |
|
|
|
<a href="https://colab.research.google.com/gist/pszemraj/4bd75aa3744f2a02a5c0ee499932b7eb/sharded-stablelm-testing-notebook.ipynb"> |
|
<img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/> |
|
</a> |
|
|
|
This is a sharded checkpoint (with ~2GB shards) of the model. Refer to the [original model](https://huggingface.co/stabilityai/stablelm-tuned-alpha-3b) for all details. |
|
|
|
|
|
## Basic Usage |
|
|
|
|
|
install `transformers`, `accelerate`, and `bitsandbytes`. |
|
|
|
```bash |
|
pip install -U -q transformers bitsandbytes accelerate |
|
``` |
|
|
|
Load the model in 8bit, then [run inference](https://huggingface.co/docs/transformers/generation_strategies#contrastive-search): |
|
|
|
```python |
|
from transformers import AutoTokenizer, AutoModelForCausalLM |
|
|
|
model_name = "ethzanalytics/stablelm-tuned-alpha-3b-sharded" |
|
tokenizer = AutoTokenizer.from_pretrained(model_name) |
|
|
|
model = AutoModelForCausalLM.from_pretrained(model_name, load_in_8bit=True) |
|
``` |
