Update README.md
Browse filesUpdate based on long-context test
README.md
CHANGED
@@ -23,7 +23,8 @@ This model was designed for roleplaying and storytelling and I think it does wel
|
|
23 |
### Long Context Tips
|
24 |
|
25 |
You can run this model past 4096 context with alpha_rope set to 1, but I think it performs better if you set alpha_rope to what you would normally use for a Llama2 model with 4096 context. For example, alpha_rope 2.5 for 8K.
|
26 |
-
Miqu can go up to 32K context in theory. I
|
|
|
27 |
|
28 |
### Sampler Tips
|
29 |
|
|
|
23 |
### Long Context Tips
|
24 |
|
25 |
You can run this model past 4096 context with alpha_rope set to 1, but I think it performs better if you set alpha_rope to what you would normally use for a Llama2 model with 4096 context. For example, alpha_rope 2.5 for 8K.
|
26 |
+
Miqu can go up to 32K context in theory. I would expect performance to degrade as you exceed 8K, which is typical for Llama2 models, but the dropoff may not be as extreme with this merge thanks to Miqu.
|
27 |
+
**UPDATE:** I was able to test my 5.0 bpw exl2 quant of this model out to 16K context just now using 8-bit cache with alpha_rope 1 and it was okay!
|
28 |
|
29 |
### Sampler Tips
|
30 |
|