Update README.md

README.md

@@ -34,6 +34,13 @@ GGML files are for CPU + GPU inference using [llama.cpp](https://github.com/gger
 * [2, 3, 4, 5, 6 and 8-bit GGML models for CPU+GPU inference](https://huggingface.co/TheBloke/guanaco-65B-GGML)
 * [Unquantised fp16 model in pytorch format, for GPU inference and for further conversions](https://huggingface.co/TheBloke/guanaco-65B-HF)
 
+## Prompt template
+
+```
+USER: prompt
+ASSISTANT:
+```
+
 <!-- compatibility_ggml start -->
 ## Compatibility
 
@@ -77,16 +84,39 @@ Refer to the Provided Files table below to see what files use which methods, and
 | guanaco-65B.ggmlv3.q5_1.bin | q5_1 | 5 | 48.97 GB | 51.47 GB | Original llama.cpp quant method, 5-bit. Even higher accuracy, resource usage and slower inference. |
 | guanaco-65B.ggmlv3.q5_K_M.bin | q5_K_M | 5 | 46.20 GB | 48.70 GB | New k-quant method. Uses GGML_TYPE_Q6_K for half of the attention.wv and feed_forward.w2 tensors, else GGML_TYPE_Q5_K |
 | guanaco-65B.ggmlv3.q5_K_S.bin | q5_K_S | 5 | 44.89 GB | 47.39 GB | New k-quant method. Uses GGML_TYPE_Q5_K for all tensors |
-
+| guanaco-65B.ggmlv3.q6_K.bin | q6_K | 6 | 53.56 GB | 56.06 GB | New k-quant method. Uses GGML_TYPE_Q8_K - 6-bit quantization - for all tensors |
+| guanaco-65B.ggmlv3.q8_0.bin | q8_0 | 8 | 69.370 GB | 71.87 GB | Original llama.cpp quant method, 8-bit. Almost indistinguishable from float16. High resource use and slow. Not recommended for most users. |
 
 **Note**: the above RAM figures assume no GPU offloading. If layers are offloaded to the GPU, this will reduce RAM usage and use VRAM instead.
 
+### q6_K and q8_0 files require expansion from archive
+
+**Note:** HF does not support uploading files larger than 50GB. Therefore I have uploaded the q6_K and q8_0 files as multi-part ZIP files. They are not compressed, it is just storing the .bin file in two parts.
+
+### q6_K 
+Please download:
+* `guanaco-65B.ggmlv3.q6_K.zip`
+* `guanaco-65B.ggmlv3.q6_K.z01`
+
+### q8_0
+Please download:
+* `guanaco-65B.ggmlv3.q8_0.zip`
+* `guanaco-65B.ggmlv3.q8_0.z01`
+
+Then extract the .zip archive. This will will expand both parts automatically. On Linux I found I had to use `7zip` - the basic `unzip` tool did not work. Example:
+```
+sudo apt update -y && sudo apt install 7zip
+7zz x guanaco-65B.ggmlv3.q6_K.zip`
+```
+
+Once the `.bin` is extracted you can delete the `.zip` and `.z01` files
+
 ## How to run in `llama.cpp`
 
 I use the following command line; adjust for your tastes and needs:
 
 ```
-./main -t 10 -ngl 32 -m guanaco-65B.ggmlv3.q5_0.bin --color -c 2048 --temp 0.7 --repeat_penalty 1.1 -n -1 -p "### Instruction: Write a story about llamas\n### Response:"
+./main -t 10 -ngl 32 -m guanaco-65B.ggmlv3.q5_0.bin --color -c 2048 --temp 0.7 --repeat_penalty 1.1 -n -1 -p "USER: Write a story about llamas\nASSISTANT:"
 ```
 Change `-t 10` to the number of physical CPU cores you have. For example if your system has 8 cores/16 threads, use `-t 8`.