|
|
--- |
|
|
license: apache-2.0 |
|
|
datasets: |
|
|
- miromind-ai/MiroVerse-v0.1 |
|
|
base_model: |
|
|
- xai-org/grok-2 |
|
|
new_version: microsoft/VibeVoice-Tokenizer |
|
|
pipeline_tag: sentence-similarity |
|
|
library_name: diffusers |
|
|
--- |
|
|
# AWS Neuron optimum model cache |
|
|
|
|
|
This repository contains cached neuron compilation artifacts for the most popular models on the Hugging Face Hub. |
|
|
|
|
|
## Inference |
|
|
|
|
|
### LLM models |
|
|
|
|
|
The transparent caching mechanism included in `optimum-neuron` and `NeuronX TGI`, makes it easier to export and deploy cached models to Neuron platforms such as Trainium and Inferentia. |
|
|
|
|
|
To deploy directly any cached model to SageMaker: |
|
|
- go to the model page, |
|
|
- select "Deploy" in the top right corner, |
|
|
- select "AWS SageMaker" in the drop-down, |
|
|
- select the "AWS Inferentia & Trainium" tab, |
|
|
- copy the code snippet. |
|
|
|
|
|
You can now paste the code snippet in your deployment script or notebook, following the instructions in the comment. |
|
|
|
|
|
To export a model to Neuron and save it locally, please follow the instructions in the `optimum-neuron` [documentation](https://huggingface.co/docs/optimum-neuron/guides/export_model). |
|
|
|
|
|
For a list of the cached models and configurations, please refer to the inference cache [configuration files](https://huggingface.co/aws-neuron/optimum-neuron-cache/tree/main/inference-cache-config). |
|
|
|
|
|
Alternatively, you can use the `optimum-cli neuron cache lookup` command to look for a specific model and see the cached configurations. |
