add files

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+*.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,17 @@
+
+---
+base_model:
+- nvidia/Llama-3.1-Nemotron-Nano-8B-v1
+---
+
+This is a converted weight from [Llama-3.1-Nemotron-Nano-8B-v1](https://huggingface.co/nvidia/Llama-3.1-Nemotron-Nano-8B-v1) model in [unsloth 4-bit dynamic quant](https://archive.is/EFz7P) using this [collab notebook](https://colab.research.google.com/drive/1P23C66j3ga49kBRnDNlmRce7R_l_-L5l?usp=sharing).
+
+## About this Conversion
+
+This conversion uses **Unsloth** to load the model in **4-bit** format and force-save it in the same **4-bit** format.
+
+### How 4-bit Quantization Works
+- The actual **4-bit quantization** is handled by **BitsAndBytes (bnb)**, which works under **Torch** via **AutoGPTQ** or **BitsAndBytes**.
+- **Unsloth** acts as a wrapper, simplifying and optimizing the process for better efficiency.
+
+This allows for reduced memory usage and faster inference while keeping the model compact.

config.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c8885c46dc593be12ff1ce21d2a4190e3a34b8d9037d71a8e0132bf002a5f1e3
+size 1378

generation_config.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5732c9b3beedbfffc3d72d5f86f8a7695e1c7a6dbefb3e611f2c749063185321
+size 235

model-00001-of-00002.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e80905a6a5098d023355e127971d64271ff64bbbd9b34b34d99bad33edca4188
+size 4652072844

model-00002-of-00002.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a7b81b0bda07ca7438c17eed8c7ede8b9a5471b63fa8a56b35444d1b0b446ce7
+size 1050673280

model.safetensors.index.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7856a93d3af2d0dcd7095869b36728e5e19e45931073c94f794db4290d3e0954
+size 132271

special_tokens_map.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f526dabe70ae46e1fb9caff406987c5e46a18f2c572906efa75b890e7833fb2a
+size 340

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6b9e4e7fb171f92fd137b777cc2714bf87d11576700a1dcd7a399e7bbe39537b
+size 17209920

tokenizer_config.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5209b74f405cfa3ef8948e1c8a053344e4ae16d1f45bc17cdc4056a3d23f3aef
+size 52644