32k-compatible

#3
by exoplanet - opened

Hi awesome bakers, great job with this (v2) reranker family. On your technical blog, "32k-compatible" is mentioned, but I couldn't figure out how this works in practice. Any pointers would be appreciated. Krispy regards.

Mixedbread org

hey @exoplanet , we've fine-tuned the model up to a context length of 8k but it generally supports 32k.

You can just pass max_length to the model during initialisation:

reranker = MxbaiRerankV2(..., max_length=32768)
juliuslipp changed discussion status to closed
Your need to confirm your account before you can post a new comment.

Sign up or log in to comment