Increasing the Context Length of the model.

#27

by siddharth-brightbots - opened Dec 12, 2024

Dec 12, 2024

I want to increase the context of the model from 32k to 128k. It states in the documentation that it supports yarn. I am using the model through serverless API. Does anyone have any idea how to increase the context length, which parameter should I change?

siddharth-brightbots changed discussion status to closed Dec 18, 2024

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

Your need to confirm your account before you can post a new comment.

· Sign up or log in to comment