sophosympatheia commited on
Commit
6179695
·
verified ·
1 Parent(s): eb8a093

Update README.md

Browse files

Update based on long-context test

Files changed (1) hide show
  1. README.md +2 -1
README.md CHANGED
@@ -23,7 +23,8 @@ This model was designed for roleplaying and storytelling and I think it does wel
23
  ### Long Context Tips
24
 
25
  You can run this model past 4096 context with alpha_rope set to 1, but I think it performs better if you set alpha_rope to what you would normally use for a Llama2 model with 4096 context. For example, alpha_rope 2.5 for 8K.
26
- Miqu can go up to 32K context in theory. I have not tested this model much past 8K right now. I would expect performance to degrade as you exceed 8K, which is typical for Llama2 models, but the dropoff may not be as extreme with this merge thanks to Miqu.
 
27
 
28
  ### Sampler Tips
29
 
 
23
  ### Long Context Tips
24
 
25
  You can run this model past 4096 context with alpha_rope set to 1, but I think it performs better if you set alpha_rope to what you would normally use for a Llama2 model with 4096 context. For example, alpha_rope 2.5 for 8K.
26
+ Miqu can go up to 32K context in theory. I would expect performance to degrade as you exceed 8K, which is typical for Llama2 models, but the dropoff may not be as extreme with this merge thanks to Miqu.
27
+ **UPDATE:** I was able to test my 5.0 bpw exl2 quant of this model out to 16K context just now using 8-bit cache with alpha_rope 1 and it was okay!
28
 
29
  ### Sampler Tips
30