Think token in vllm
#1
by
sbrnaderi
- opened
I following the instructions in the model card, and run this model with vllm, but I am getting think tokens like what you see in the example below. Is this expected? or am I doing something wrong?
<seed:think>
Got it, the user just said "hi". I should respond in a friendly and open way to encourage them to say more. Maybe keep it simple and welcoming. Let me go with something like, "Hi there! How can I help you today?" That's friendly and invites them to share what they need.</seed:think>Hi there! How can I help you today?
Those were the correct results. You can find them in tokenizer_config.json
cheers
ah, thanks for your response and also for making this AWQ quant.
Cheers