File size: 551 Bytes
78075a8 a6fc6fe 76e1ab9 e77ec90 a6fc6fe 768f5de 76e1ab9 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
---
license: mit
datasets:
- Open-Orca/OpenOrca
language:
- en
library_name: transformers
pipeline_tag: text-generation
---
```
model_name = "circulus/Llama-2-13b-orca-v1"
tokenizer = AutoTokenizer.from_pretrained(model_name, use_fast=False)
config = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_compute_dtype=torch.bfloat16, bnb_4bit_use_double_quant=True)
model = AutoModelForCausalLM.from_pretrained(model_name, device_map="auto", quantization_config=config)
``` |