Longest Context is 512？
#1
by
Pony
- opened
What is the maximum length of context that can be handled?
For our open-sourced checkpoint, yes, 512.
If we want 1024 or more, we can retrain the model with more memory token buckets.