neuralmagic
/

Mistral-7B-Instruct-v0.3-quantized.w8a8

Text Generation

text-generation-inference

Inference Endpoints

8-bit precision

compressed-tensors

Model card Files Files and versions Community

README.md exists but content is empty.

Downloads last month: 210

Safetensors

Model size

7.25B params

Tensor type

BF16

·

I8

·

Inference Providers NEW

Text Generation

This model is not currently available via any of the supported Inference Providers.

Collection including neuralmagic/Mistral-7B-Instruct-v0.3-quantized.w8a8

INT8 LLMs for vLLM

Accurate INT8 quantized models by Neural Magic, ready for use with vLLM! • 50 items • Updated Sep 26, 2024 • 15