---
library_name: keras-hub
---
### Model Overview
# Model Summary

Mistral is a set of large language models published by the Mistral AI team. The Mixtral-8x7B Large Language Model (LLM) is a pretrained generative Sparse Mixture of Experts. Both pre-trained and instruction tuned models are available with 7 billion activated parameters. 

Weights are released under the [Apache 2 License](https://github.com/keras-team/keras-hub/blob/master/LICENSE) . Keras model code is released under the [Apache 2 License](https://github.com/keras-team/keras-hub/blob/master/LICENSE).

## Links

* [Mixtral Quickstart Notebook](https://www.kaggle.com/code/laxmareddypatlolla/mixtral-quickstart-notebook)
* [Mixtral API Documentation](https://keras.io/keras_hub/api/models/mixtral/)
* [Mixtral Model Card](https://mistral.ai/news/mixtral-of-experts)
* [KerasHub Beginner Guide](https://keras.io/guides/keras_hub/getting_started/)
* [KerasHub Model Publishing Guide](https://keras.io/guides/keras_hub/upload/)

## Installation

Keras and KerasHub can be installed with:

```
pip install -U -q keras-hub
pip install -U -q keras
```

Jax, TensorFlow, and Torch come preinstalled in Kaggle Notebooks. For instructions on installing them in another environment see the [Keras Getting Started](https://keras.io/getting_started/) page.

## Presets

The following model checkpoints are provided by the Keras team. Full code examples for each are available below.

| Preset name                            | Parameters | Description                                                                                                  |
|---------------------------------------|------------|--------------------------------------------------------------------------------------------------------------|
| mixtral_8_7b_en       | 7B      | 32-layer Mixtral MoE model with 7 billion active parameters and 8 experts per MoE layer. |
| mixtral_8_instruct_7b_en    | 7B      | Instruction fine-tuned 32-layer Mixtral MoE model with 7 billion active parameters and 8 experts per MoE layer. |

## Example Usage
```Python

import keras
import keras_hub
import numpy as np

# Basic text generation
mixtral_lm = keras_hub.models.MixtralCausalLM.from_preset("mixtral_8_7b_en")
mixtral_lm.generate("[INST] What is Keras? [/INST]", max_length=500)

# Generate with batched prompts
mixtral_lm.generate([
    "[INST] What is Keras? [/INST]",
    "[INST] Give me your best brownie recipe. [/INST]"
], max_length=500)

# Using different sampling strategies
mixtral_lm = keras_hub.models.MixtralCausalLM.from_preset("mixtral_8_7b_en")
# Greedy sampling
mixtral_lm.compile(sampler="greedy")
mixtral_lm.generate("I want to say", max_length=30)

# Beam search
mixtral_lm.compile(
    sampler=keras_hub.samplers.BeamSampler(
        num_beams=2,
        top_k_experts=2,  # MoE-specific: number of experts to use per token
    )
)
mixtral_lm.generate("I want to say", max_length=30)

# Generate without preprocessing
prompt = {
    "token_ids": np.array([[1, 315, 947, 298, 1315, 0, 0, 0, 0, 0]] * 2),
    "padding_mask": np.array([[1, 1, 1, 1, 1, 0, 0, 0, 0, 0]] * 2),
}

mixtral_lm = keras_hub.models.MixtralCausalLM.from_preset(
    "mixtral_8_7b_en",
    preprocessor=None,
    dtype="bfloat16"
)
mixtral_lm.generate(
    prompt,
    num_experts=8,           # Total number of experts per layer
    top_k_experts=2,         # Number of experts to use per token
    router_aux_loss_coef=0.02  # Router auxiliary loss coefficient
)

# Training on a single batch
features = ["The quick brown fox jumped.", "I forgot my homework."]
mixtral_lm = keras_hub.models.MixtralCausalLM.from_preset(
    "mixtral_8_7b_en", 
    dtype="bfloat16"
)
mixtral_lm.fit(
    x=features, 
    batch_size=2,
    router_aux_loss_coef=0.02  # MoE-specific: router training loss
)

# Training without preprocessing
x = {
    "token_ids": np.array([[1, 315, 947, 298, 1315, 369, 315, 837, 0, 0]] * 2),
    "padding_mask": np.array([[1, 1, 1, 1, 1, 1, 1, 1, 0, 0]] * 2),
}
y = np.array([[315, 947, 298, 1315, 369, 315, 837, 0, 0, 0]] * 2)
sw = np.array([[1, 1, 1, 1, 1, 1, 1, 0, 0, 0]] * 2)

mixtral_lm = keras_hub.models.MixtralCausalLM.from_preset(
    "mixtral_8_7b_en",
    preprocessor=None,
    dtype="bfloat16"
)
mixtral_lm.fit(
    x=x,
    y=y,
    sample_weight=sw,
    batch_size=2,
    router_aux_loss_coef=0.02
)
```

## Example Usage with Hugging Face URI

```Python

import keras
import keras_hub
import numpy as np

# Basic text generation
mixtral_lm = keras_hub.models.MixtralCausalLM.from_preset("hf://keras/mixtral_8_7b_en")
mixtral_lm.generate("[INST] What is Keras? [/INST]", max_length=500)

# Generate with batched prompts
mixtral_lm.generate([
    "[INST] What is Keras? [/INST]",
    "[INST] Give me your best brownie recipe. [/INST]"
], max_length=500)

# Using different sampling strategies
mixtral_lm = keras_hub.models.MixtralCausalLM.from_preset("hf://keras/mixtral_8_7b_en")
# Greedy sampling
mixtral_lm.compile(sampler="greedy")
mixtral_lm.generate("I want to say", max_length=30)

# Beam search
mixtral_lm.compile(
    sampler=keras_hub.samplers.BeamSampler(
        num_beams=2,
        top_k_experts=2,  # MoE-specific: number of experts to use per token
    )
)
mixtral_lm.generate("I want to say", max_length=30)

# Generate without preprocessing
prompt = {
    "token_ids": np.array([[1, 315, 947, 298, 1315, 0, 0, 0, 0, 0]] * 2),
    "padding_mask": np.array([[1, 1, 1, 1, 1, 0, 0, 0, 0, 0]] * 2),
}

mixtral_lm = keras_hub.models.MixtralCausalLM.from_preset(
    "hf://keras/mixtral_8_7b_en",
    preprocessor=None,
    dtype="bfloat16"
)
mixtral_lm.generate(
    prompt,
    num_experts=8,           # Total number of experts per layer
    top_k_experts=2,         # Number of experts to use per token
    router_aux_loss_coef=0.02  # Router auxiliary loss coefficient
)

# Training on a single batch
features = ["The quick brown fox jumped.", "I forgot my homework."]
mixtral_lm = keras_hub.models.MixtralCausalLM.from_preset(
    "hf://keras/mixtral_8_7b_en", 
    dtype="bfloat16"
)
mixtral_lm.fit(
    x=features, 
    batch_size=2,
    router_aux_loss_coef=0.02  # MoE-specific: router training loss
)

# Training without preprocessing
x = {
    "token_ids": np.array([[1, 315, 947, 298, 1315, 369, 315, 837, 0, 0]] * 2),
    "padding_mask": np.array([[1, 1, 1, 1, 1, 1, 1, 1, 0, 0]] * 2),
}
y = np.array([[315, 947, 298, 1315, 369, 315, 837, 0, 0, 0]] * 2)
sw = np.array([[1, 1, 1, 1, 1, 1, 1, 0, 0, 0]] * 2)

mixtral_lm = keras_hub.models.MixtralCausalLM.from_preset(
    "hf://keras/mixtral_8_7b_en",
    preprocessor=None,
    dtype="bfloat16"
)
mixtral_lm.fit(
    x=x,
    y=y,
    sample_weight=sw,
    batch_size=2,
    router_aux_loss_coef=0.02
)
```