Hugging Face's logo

medmekk
/
SmolLM2-1.7B-Instruct.GGUF

GGUF
imatrix
conversational
Model card Files Community
medmekk HF Staff commited on
Commit
01d16a0
·
verified ·
1 Parent(s): d925b88

Upload quantized models

Browse files
Files changed (22) hide show
  1. .gitattributes +20 -0
  2. README.md +34 -0
  3. SmolLM2-1.7B-Instruct-IQ3_M_imat.gguf +3 -0
  4. SmolLM2-1.7B-Instruct-IQ3_XXS_imat.gguf +3 -0
  5. SmolLM2-1.7B-Instruct-IQ4_NL_imat.gguf +3 -0
  6. SmolLM2-1.7B-Instruct-IQ4_XS_imat.gguf +3 -0
  7. SmolLM2-1.7B-Instruct-Q2_K.gguf +3 -0
  8. SmolLM2-1.7B-Instruct-Q3_K_L.gguf +3 -0
  9. SmolLM2-1.7B-Instruct-Q3_K_M.gguf +3 -0
  10. SmolLM2-1.7B-Instruct-Q3_K_S.gguf +3 -0
  11. SmolLM2-1.7B-Instruct-Q4_0.gguf +3 -0
  12. SmolLM2-1.7B-Instruct-Q4_K_M.gguf +3 -0
  13. SmolLM2-1.7B-Instruct-Q4_K_M_imat.gguf +3 -0
  14. SmolLM2-1.7B-Instruct-Q4_K_S.gguf +3 -0
  15. SmolLM2-1.7B-Instruct-Q4_K_S_imat.gguf +3 -0
  16. SmolLM2-1.7B-Instruct-Q5_0.gguf +3 -0
  17. SmolLM2-1.7B-Instruct-Q5_K_M.gguf +3 -0
  18. SmolLM2-1.7B-Instruct-Q5_K_M_imat.gguf +3 -0
  19. SmolLM2-1.7B-Instruct-Q5_K_S.gguf +3 -0
  20. SmolLM2-1.7B-Instruct-Q5_K_S_imat.gguf +3 -0
  21. SmolLM2-1.7B-Instruct-Q6_K.gguf +3 -0
  22. SmolLM2-1.7B-Instruct-Q8_0.gguf +3 -0
.gitattributes CHANGED
@@ -33,3 +33,23 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ SmolLM2-1.7B-Instruct-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
37
+ SmolLM2-1.7B-Instruct-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
38
+ SmolLM2-1.7B-Instruct-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
39
+ SmolLM2-1.7B-Instruct-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
40
+ SmolLM2-1.7B-Instruct-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
41
+ SmolLM2-1.7B-Instruct-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
42
+ SmolLM2-1.7B-Instruct-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
43
+ SmolLM2-1.7B-Instruct-Q5_0.gguf filter=lfs diff=lfs merge=lfs -text
44
+ SmolLM2-1.7B-Instruct-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
45
+ SmolLM2-1.7B-Instruct-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
46
+ SmolLM2-1.7B-Instruct-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
47
+ SmolLM2-1.7B-Instruct-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
48
+ SmolLM2-1.7B-Instruct-IQ3_M_imat.gguf filter=lfs diff=lfs merge=lfs -text
49
+ SmolLM2-1.7B-Instruct-IQ3_XXS_imat.gguf filter=lfs diff=lfs merge=lfs -text
50
+ SmolLM2-1.7B-Instruct-Q4_K_M_imat.gguf filter=lfs diff=lfs merge=lfs -text
51
+ SmolLM2-1.7B-Instruct-Q4_K_S_imat.gguf filter=lfs diff=lfs merge=lfs -text
52
+ SmolLM2-1.7B-Instruct-IQ4_NL_imat.gguf filter=lfs diff=lfs merge=lfs -text
53
+ SmolLM2-1.7B-Instruct-IQ4_XS_imat.gguf filter=lfs diff=lfs merge=lfs -text
54
+ SmolLM2-1.7B-Instruct-Q5_K_M_imat.gguf filter=lfs diff=lfs merge=lfs -text
55
+ SmolLM2-1.7B-Instruct-Q5_K_S_imat.gguf filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,34 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+
2
+ # medmekk/SmolLM2-1.7B-Instruct.GGUF
3
+ GGUF quantized versions of [HuggingFaceTB/SmolLM2-1.7B-Instruct](https://huggingface.co/HuggingFaceTB/SmolLM2-1.7B-Instruct)
4
+
5
+ ## Available Formats:
6
+ - `Q2_K`: SmolLM2-1.7B-Instruct-Q2_K.gguf
7
+ - `Q3_K_S`: SmolLM2-1.7B-Instruct-Q3_K_S.gguf
8
+ - `Q3_K_M`: SmolLM2-1.7B-Instruct-Q3_K_M.gguf
9
+ - `Q3_K_L`: SmolLM2-1.7B-Instruct-Q3_K_L.gguf
10
+ - `Q4_0`: SmolLM2-1.7B-Instruct-Q4_0.gguf
11
+ - `Q4_K_S`: SmolLM2-1.7B-Instruct-Q4_K_S.gguf
12
+ - `Q4_K_M`: SmolLM2-1.7B-Instruct-Q4_K_M.gguf
13
+ - `Q5_0`: SmolLM2-1.7B-Instruct-Q5_0.gguf
14
+ - `Q5_K_S`: SmolLM2-1.7B-Instruct-Q5_K_S.gguf
15
+ - `Q5_K_M`: SmolLM2-1.7B-Instruct-Q5_K_M.gguf
16
+ - `Q6_K`: SmolLM2-1.7B-Instruct-Q6_K.gguf
17
+ - `Q8_0`: SmolLM2-1.7B-Instruct-Q8_0.gguf
18
+ - `IQ3_M_IMAT`: SmolLM2-1.7B-Instruct-IQ3_M_imat.gguf
19
+ - `IQ3_XXS_IMAT`: SmolLM2-1.7B-Instruct-IQ3_XXS_imat.gguf
20
+ - `Q4_K_M_IMAT`: SmolLM2-1.7B-Instruct-Q4_K_M_imat.gguf
21
+ - `Q4_K_S_IMAT`: SmolLM2-1.7B-Instruct-Q4_K_S_imat.gguf
22
+ - `IQ4_NL_IMAT`: SmolLM2-1.7B-Instruct-IQ4_NL_imat.gguf
23
+ - `IQ4_XS_IMAT`: SmolLM2-1.7B-Instruct-IQ4_XS_imat.gguf
24
+ - `Q5_K_M_IMAT`: SmolLM2-1.7B-Instruct-Q5_K_M_imat.gguf
25
+ - `Q5_K_S_IMAT`: SmolLM2-1.7B-Instruct-Q5_K_S_imat.gguf
26
+
27
+ ## Usage with llama.cpp:
28
+ ```bash
29
+ # CLI:
30
+ llama-cli --hf-repo medmekk/SmolLM2-1.7B-Instruct.GGUF --hf-file MODEL_FILE -p "Your prompt"
31
+
32
+ # Server:
33
+ llama-server --hf-repo medmekk/SmolLM2-1.7B-Instruct.GGUF --hf-file MODEL_FILE -c 2048
34
+ ```
SmolLM2-1.7B-Instruct-IQ3_M_imat.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5ec8518ee24be867afe18db2582a6494e024f332d25bf462a3842b18704d4fd6
3
+ size 810243264
SmolLM2-1.7B-Instruct-IQ3_XXS_imat.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f8ba1d6c4678d690139d04ba56838a3909cb4380eaa9ed5338366c2c911f5812
3
+ size 680088768
SmolLM2-1.7B-Instruct-IQ4_NL_imat.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0982d3310d504dcd17c8dadebdc0b9c1a4380f1c53a2253a3bce07b63e1220d8
3
+ size 990729408
SmolLM2-1.7B-Instruct-IQ4_XS_imat.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:482808749c4bdbe1fb8fa983784fb8d331fc991168fc73c5a823f66d54e4157b
3
+ size 940397760
SmolLM2-1.7B-Instruct-Q2_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:99fe8fa0a755846dc85e122ded785cdfde75b6ac31c27a0b53f062974174aa65
3
+ size 674583520
SmolLM2-1.7B-Instruct-Q3_K_L.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fb73537cb3bde7cce0073af302050b1913b216da348c6e00a3fdd209d289a84c
3
+ size 932533216
SmolLM2-1.7B-Instruct-Q3_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:aece2d02dc89a8dc0f33b1bf47ad0db32b2d3e5462bea16a48c7d90ead239c83
3
+ size 860181472
SmolLM2-1.7B-Instruct-Q3_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:037dab95871a719f2e03d1806110de2a1689ec08056d06e990ec4050b9f8a233
3
+ size 776819680
SmolLM2-1.7B-Instruct-Q4_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4470d5f42c153a139f62ab2c7b21f135d74c66a73045160c010e3cd46a5c8228
3
+ size 990729184
SmolLM2-1.7B-Instruct-Q4_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f4edae912de37bd3b326e2757292f71aa02653c38442da5896ff198687586cce
3
+ size 1055609824
SmolLM2-1.7B-Instruct-Q4_K_M_imat.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:de8966225f5e69e7ab579ec9627e704cf17cb799be33d67dcfbb7f114843ca62
3
+ size 1055610048
SmolLM2-1.7B-Instruct-Q4_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:805d5bb771c33af19409ce9d99c3fa9c1b9569046a5105e478837337187e3866
3
+ size 999117792
SmolLM2-1.7B-Instruct-Q4_K_S_imat.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:bb03ca65ae706be9f93261eef2e06d50af2bc38ccd824f253531d64e577c6729
3
+ size 999118016
SmolLM2-1.7B-Instruct-Q5_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0198c0f95da8e843bb0cdf873bb8480142284f2df8a5aab11b2d5e3d2a745ca8
3
+ size 1192055776
SmolLM2-1.7B-Instruct-Q5_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9d4565b0ce89b9a4b60b5863fb83cb0bc3247f6c8217271340cd598a96328750
3
+ size 1225479136
SmolLM2-1.7B-Instruct-Q5_K_M_imat.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d404e4c28c40f7ed2cbae561ddca16486891a591500c80934f40a9b9419ca0d3
3
+ size 1225479360
SmolLM2-1.7B-Instruct-Q5_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0fd16b1a51a6adbf1606dde4b945e4c52b5b37719d25886e0d94aad838263e54
3
+ size 1192055776
SmolLM2-1.7B-Instruct-Q5_K_S_imat.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e52f4a5eb81dc9ea77ed2820f5b11bdc2c45329cf1db329d0b96121585df85fe
3
+ size 1192056000
SmolLM2-1.7B-Instruct-Q6_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:745025c86d69519e5f28b0ec699f0b884b2f4dca96ef936c67f065ef39d07ef0
3
+ size 1405965280
SmolLM2-1.7B-Instruct-Q8_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:79f6515d925a520486c54128c7ccb08fc33b5b124b93c93e595341d7ac2eedc7
3
+ size 1820414944