---
license: apache-2.0
datasets:
- openerotica/mixed-rp
- kingbri/PIPPA-shareGPT
language:
- en
base_model:
- N-Bot-Int/OpenElla3-Llama3.2B
pipeline_tag: text-generation
library_name: transformers
tags:
- unsloth
- Uncensored
- text-generation-inference
- transformers
- unsloth
- llama
- trl
- roleplay
- conversational
new_version: N-Bot-Int/OpenElla3-Llama3.2B-GGUF
---
<a href="https://imgbb.com/"><img src="https://raw.githubusercontent.com/ItsMeDevRoland/NexusBotWorkInteractives/refs/heads/main/1741962070023.png" alt="1741962070023" border="0"></a>
# GGUF Version
  **GGUF** with Quants! Allowing you to run models using KoboldCPP and other AI Environments!


# Quantizations:
| Quant Type    | Benefits                                          | Cons                                              |
|---------------|---------------------------------------------------|---------------------------------------------------|
| **Q4_K_M**    | ✅ Smallest size (fastest inference)              | ❌ Lowest accuracy compared to other quants      |
|               | ✅ Requires the least VRAM/RAM                    | ❌ May struggle with complex reasoning           |
|               | ✅ Ideal for edge devices & low-resource setups   | ❌ Can produce slightly degraded text quality    |
| **Q5_K_M**    | ✅ Better accuracy than Q4, while still compact   | ❌ Slightly larger model size than Q4            |
|               | ✅ Good balance between speed and precision       | ❌ Needs a bit more VRAM than Q4                 |
|               | ✅ Works well on mid-range GPUs                   | ❌ Still not as accurate as higher-bit models    |
| **Q8_0**      | ✅ Highest accuracy (closest to full model)       | ❌ Requires significantly more VRAM/RAM          |
|               | ✅ Best for complex reasoning & detailed outputs  | ❌ Slower inference compared to Q4 & Q5          |
|               | ✅ Suitable for high-end GPUs & serious workloads | ❌ Larger file size (takes more storage)         |

# Model Details:
  Read the Model details on huggingface
  <link src="https://huggingface.co/N-Bot-Int/OpenElla3-Llama3.2">Model Detail Here!</link>