StableLM-Base-Alpha: sharded checkpoint
This is a sharded checkpoint (with ~2GB shards) of the model. Refer to the original model for all details.
Basic Usage
install transformers, accelerate, and bitsandbytes.
pip install -U -q transformers bitsandbytes accelerate
Load the model in 8bit, then run inference:
from transformers import AutoTokenizer, AutoModelForCausalLM
model_name = "ethzanalytics/stablelm-base-alpha-3b-sharded"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name, load_in_8bit=True)
- Downloads last month
- 27
Inference Providers
NEW
This model is not currently available via any of the supported third-party Inference Providers, and
the model is not deployed on the HF Inference API.