metadata
tags:
- Llamba
- recurrent-models
- distillation
- cartesia
- edge
license: apache-2.0
library_name: cartesia-pytorch
datasets:
- ai2_arc
- PIQA
- Winogrande
- HellaSwag
- Lambada
- MMLU
- OpenBookQA
inference:
precision: bf16
hardware: gpu
Llamba Models
The Llamba models are part of Cartesia's Edge library, designed for efficient, high-performance machine learning applications.
For more details, refer to the paper.
Usage
Llamba on PyTorch
To use Llamba with PyTorch:
- Install the required package:
pip install --no-binary :all: cartesia-pytorch
- Load and run the model
from transformers import AutoTokenizer
from cartesia_pytorch.Llamba.llamba import LlambaLMHeadModel
model = LlambaLMHeadModel.from_pretrained("cartesia-ai/Llamba-8B", strict=True).to('cuda')
tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-3.1-8B")
input_ids = tokenizer("Hello, my name is", return_tensors="pt").input_ids
input_ids = input_ids.to('cuda')
output = model.generate(input_ids, max_length=100)[0]
print(tokenizer.decode(output, skip_special_tokens=True))
Llamba on MLX
To run Llamba with the Metal framework see cartesia-metal
Evaluations
The Llamba models have been evaluated on multiple standard benchmarks, demonstrating efficiency gains while maintaining strong performance. Below are the results:
Model | ARC-C (0-shot) | ARC-C (25-shot) | ARC-E (0-shot) | ARC-E (25-shot) | PIQA (0-shot) | PIQA (10-shot) | WG (0-shot) | WG (5-shot) |
---|---|---|---|---|---|---|---|---|
Llamba-1B | 37.2 | 41.8 | 69.5 | 71.2 | 74.0 | 74.3 | 60.6 | 58.1 |
Llamba-3B | 48.5 | 53.0 | 79.0 | 81.1 | 78.6 | 79.5 | 70.4 | 72.4 |
Llamba-8B | 54.6 | 60.0 | 82.5 | 85.8 | 80.9 | 81.5 | 73.3 | 76.9 |
Model | HS (0-shot) | HS (10-shot) | LMB (0-shot) | LMB (10-shot) | MMLU (0-shot) | MMLU (5-shot) | OBQA (0-shot) | OBQA (10-shot) |
---|---|---|---|---|---|---|---|---|
Llamba-1B | 61.2 | 60.2 | 48.4 | 39.0 | 38.0 | 31.3 | 37.0 | 38.0 |
Llamba-3B | 73.8 | 74.3 | 65.8 | 60.0 | 52.7 | 50.3 | 42.8 | 42.8 |
Llamba-8B | 77.6 | 78.7 | 69.4 | 65.0 | 61.0 | 60.0 | 43.4 | 45.8 |
More details on model performance, benchmarks, and evaluation metrics can be found in the paper.