Upload 7 files

.gitattributes

@@ -33,3 +33,9 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+WizardLM-2-7B-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text
+WizardLM-2-7B-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text
+WizardLM-2-7B-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text
+WizardLM-2-7B-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text
+WizardLM-2-7B-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text
+WizardLM-2-7B-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,30 @@
+---
+license: other
+language:
+- en
+pipeline_tag: text-generation
+tags:
+- gguf
+- imatrix
+- wizardlm
+---
+Quantizations of https://huggingface.co/dreamgen/WizardLM-2-7B
+
+# From original readme
+
+## Usage
+
+❗<b>Note for model system prompts usage:</b>
+
+
+<b>WizardLM-2</b>  adopts the prompt format from <b>Vicuna</b> and supports **multi-turn** conversation. The prompt should be as following:
+
+```
+A chat between a curious user and an artificial intelligence assistant. The assistant gives helpful, 
+detailed, and polite answers to the user's questions. USER: Hi ASSISTANT: Hello.</s>
+USER: Who are you? ASSISTANT: I am WizardLM.</s>......
+```
+
+<b> Inference WizardLM-2 Demo Script</b>
+
+We provide a WizardLM-2 inference demo [code](https://github.com/nlpxucan/WizardLM/tree/main/demo) on our github.

WizardLM-2-7B-IQ1_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5e1792098390bb0f509208db447d55f2866bd5895fe164f96d4cfc3eb18c58be
+size 1754445504

WizardLM-2-7B-IQ1_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6b2d0c8d4b622ff64356361fd3234fd3c6972376681d6d4cda5bd4730d21be00
+size 1612101312

WizardLM-2-7B-IQ2_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1b158a71ec17692d5b15834cf33fb61a6e7ca30ba7e7dee4f416f016d1a8ec28
+size 2500712128

WizardLM-2-7B-IQ2_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:53f99906f68abaf1d98f9decff77ec3684fb7a5e1508dca28b64767fdb8e1eed
+size 2310919872

WizardLM-2-7B-IQ2_XS.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8207fae8aa4b89033922badc2d44300ceafba72f4da823a8284d0f991cd259f1
+size 2198255296

WizardLM-2-7B-IQ2_XXS.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bebf2b54790fc46315db45812cdb46801ab210714b92dabb247a7f72747d2d46
+size 1991685824