Hugging Face's logo

medmekk
/
gemma-2-2b-it.GGUF

GGUF
Inference Endpoints
imatrix
conversational
Model card Files Community
medmekk HF staff commited on
Commit
8ff9c52
·
verified ·
1 Parent(s): d242773

Upload quantized models

Browse files
Files changed (22) hide show
  1. .gitattributes +20 -0
  2. README.md +34 -0
  3. gemma-2-2b-it-IQ3_M_imat.gguf +3 -0
  4. gemma-2-2b-it-IQ3_XXS_imat.gguf +3 -0
  5. gemma-2-2b-it-IQ4_NL_imat.gguf +3 -0
  6. gemma-2-2b-it-IQ4_XS_imat.gguf +3 -0
  7. gemma-2-2b-it-Q2_K.gguf +3 -0
  8. gemma-2-2b-it-Q3_K_L.gguf +3 -0
  9. gemma-2-2b-it-Q3_K_M.gguf +3 -0
  10. gemma-2-2b-it-Q3_K_S.gguf +3 -0
  11. gemma-2-2b-it-Q4_0.gguf +3 -0
  12. gemma-2-2b-it-Q4_K_M.gguf +3 -0
  13. gemma-2-2b-it-Q4_K_M_imat.gguf +3 -0
  14. gemma-2-2b-it-Q4_K_S.gguf +3 -0
  15. gemma-2-2b-it-Q4_K_S_imat.gguf +3 -0
  16. gemma-2-2b-it-Q5_0.gguf +3 -0
  17. gemma-2-2b-it-Q5_K_M.gguf +3 -0
  18. gemma-2-2b-it-Q5_K_M_imat.gguf +3 -0
  19. gemma-2-2b-it-Q5_K_S.gguf +3 -0
  20. gemma-2-2b-it-Q5_K_S_imat.gguf +3 -0
  21. gemma-2-2b-it-Q6_K.gguf +3 -0
  22. gemma-2-2b-it-Q8_0.gguf +3 -0
.gitattributes CHANGED
@@ -33,3 +33,23 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ gemma-2-2b-it-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
37
+ gemma-2-2b-it-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
38
+ gemma-2-2b-it-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
39
+ gemma-2-2b-it-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
40
+ gemma-2-2b-it-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
41
+ gemma-2-2b-it-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
42
+ gemma-2-2b-it-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
43
+ gemma-2-2b-it-Q5_0.gguf filter=lfs diff=lfs merge=lfs -text
44
+ gemma-2-2b-it-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
45
+ gemma-2-2b-it-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
46
+ gemma-2-2b-it-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
47
+ gemma-2-2b-it-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
48
+ gemma-2-2b-it-IQ3_M_imat.gguf filter=lfs diff=lfs merge=lfs -text
49
+ gemma-2-2b-it-IQ3_XXS_imat.gguf filter=lfs diff=lfs merge=lfs -text
50
+ gemma-2-2b-it-Q4_K_M_imat.gguf filter=lfs diff=lfs merge=lfs -text
51
+ gemma-2-2b-it-Q4_K_S_imat.gguf filter=lfs diff=lfs merge=lfs -text
52
+ gemma-2-2b-it-IQ4_NL_imat.gguf filter=lfs diff=lfs merge=lfs -text
53
+ gemma-2-2b-it-IQ4_XS_imat.gguf filter=lfs diff=lfs merge=lfs -text
54
+ gemma-2-2b-it-Q5_K_M_imat.gguf filter=lfs diff=lfs merge=lfs -text
55
+ gemma-2-2b-it-Q5_K_S_imat.gguf filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,34 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+
2
+ # medmekk/gemma-2-2b-it.GGUF
3
+ GGUF quantized versions of [google/gemma-2-2b-it](https://huggingface.co/google/gemma-2-2b-it)
4
+
5
+ ## Available Formats:
6
+ - `Q2_K`: gemma-2-2b-it-Q2_K.gguf
7
+ - `Q3_K_S`: gemma-2-2b-it-Q3_K_S.gguf
8
+ - `Q3_K_M`: gemma-2-2b-it-Q3_K_M.gguf
9
+ - `Q3_K_L`: gemma-2-2b-it-Q3_K_L.gguf
10
+ - `Q4_0`: gemma-2-2b-it-Q4_0.gguf
11
+ - `Q4_K_S`: gemma-2-2b-it-Q4_K_S.gguf
12
+ - `Q4_K_M`: gemma-2-2b-it-Q4_K_M.gguf
13
+ - `Q5_0`: gemma-2-2b-it-Q5_0.gguf
14
+ - `Q5_K_S`: gemma-2-2b-it-Q5_K_S.gguf
15
+ - `Q5_K_M`: gemma-2-2b-it-Q5_K_M.gguf
16
+ - `Q6_K`: gemma-2-2b-it-Q6_K.gguf
17
+ - `Q8_0`: gemma-2-2b-it-Q8_0.gguf
18
+ - `IQ3_M_IMAT`: gemma-2-2b-it-IQ3_M_imat.gguf
19
+ - `IQ3_XXS_IMAT`: gemma-2-2b-it-IQ3_XXS_imat.gguf
20
+ - `Q4_K_M_IMAT`: gemma-2-2b-it-Q4_K_M_imat.gguf
21
+ - `Q4_K_S_IMAT`: gemma-2-2b-it-Q4_K_S_imat.gguf
22
+ - `IQ4_NL_IMAT`: gemma-2-2b-it-IQ4_NL_imat.gguf
23
+ - `IQ4_XS_IMAT`: gemma-2-2b-it-IQ4_XS_imat.gguf
24
+ - `Q5_K_M_IMAT`: gemma-2-2b-it-Q5_K_M_imat.gguf
25
+ - `Q5_K_S_IMAT`: gemma-2-2b-it-Q5_K_S_imat.gguf
26
+
27
+ ## Usage with llama.cpp:
28
+ ```bash
29
+ # CLI:
30
+ llama-cli --hf-repo medmekk/gemma-2-2b-it.GGUF --hf-file MODEL_FILE -p "Your prompt"
31
+
32
+ # Server:
33
+ llama-server --hf-repo medmekk/gemma-2-2b-it.GGUF --hf-file MODEL_FILE -c 2048
34
+ ```
gemma-2-2b-it-IQ3_M_imat.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ad818aa7c970b457880b6827e83d8e4dfaa6958c7cad73f14330cebe0fe82c5e
3
+ size 1393561664
gemma-2-2b-it-IQ3_XXS_imat.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9d991b91e3ca6897f83a466b2c454b7ef56b5194321e81d79f564f16e842ab34
3
+ size 1181685824
gemma-2-2b-it-IQ4_NL_imat.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7259bdb4f1ac946b29f0d347d5f1760a4a5bc21f8905f294fb4a0feea9a50b0e
3
+ size 1629509696
gemma-2-2b-it-IQ4_XS_imat.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f6903689917e71e71158b1f2193f3aacd4b00e298841693743d9b767d212c410
3
+ size 1566251072
gemma-2-2b-it-Q2_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f8435f409d1cf75e2dd7d463acbc3b76fa560aa7627e171efdada2cccec26460
3
+ size 1229829984
gemma-2-2b-it-Q3_K_L.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2974d6e9081186ea3f327adf7f575ef4037ebbe9abf5e3d009f72591cd994b38
3
+ size 1550436192
gemma-2-2b-it-Q3_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:644d3d4d414f5ed169c4aa05a88395c64f146c909d97ae4569f228012798cdbb
3
+ size 1461667680
gemma-2-2b-it-Q3_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1d905176fba6fc801341a5e3e4b12b920dd0275992f50c1e55e96d44abce7bdc
3
+ size 1360660320
gemma-2-2b-it-Q4_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:596488296ac9de4a2a2d53522cd2c744d835017c9321a39ee8e0d3ccc0c8c512
3
+ size 1629509472
gemma-2-2b-it-Q4_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a9de40ca88a0f9a0c229d8ee97f9db1a4f533ef5d9aa955cb6cfba814fb96b29
3
+ size 1708582752
gemma-2-2b-it-Q4_K_M_imat.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:302f03ac232df49813f2dd575880fb83f76a1a982731495f1b4eadda6369dfdc
3
+ size 1708582976
gemma-2-2b-it-Q4_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3f9f5df757c30fc59856949e519e48566f087ec0c774a3e69f5aee1f9420e22f
3
+ size 1638651744
gemma-2-2b-it-Q4_K_S_imat.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1e8f1d1e5958fcd3870d904a74ce90a2c0fd210de5cc7408aef526756f9e133d
3
+ size 1638651968
gemma-2-2b-it-Q5_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:14d9340760eeaa35a7266a5ad50abf30a434ff3788a932b3cc14d916faaa3eb4
3
+ size 1882543968
gemma-2-2b-it-Q5_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1b514d24747a725660e84ed7778dc0a374fd1155659cd8e1064e57882d57d786
3
+ size 1923278688
gemma-2-2b-it-Q5_K_M_imat.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5f1b2264be651dfc53947d76f7f475ab18a1e7b255b5a23a9a2970178520c115
3
+ size 1923278912
gemma-2-2b-it-Q5_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e50f490839ae38697165464e6b459966ac70d66c62160cb5778ac5c70357e58b
3
+ size 1882543968
gemma-2-2b-it-Q5_K_S_imat.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:701fb1c775f4c21279e5b72eea7de440126dbd5f73c4f23459be3dda6a209c14
3
+ size 1882544192
gemma-2-2b-it-Q6_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8a32524a1eec1931ebafb5ed24f19dcc2561b1c86d3fa02a578fca4bd34def1c
3
+ size 2151393120
gemma-2-2b-it-Q8_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0d69e1c933f8297fc3e5d915ac964cb37f9b19ffa42392c81bdfaa37d770d6ee
3
+ size 2784495456