fix: remove `add_generation_prompt` from H1 tokenizers
#6
by
ybelkada
- opened
add_generation_prompt is set to False by default on HF: https://github.com/huggingface/transformers/blob/fe5f3c85d292e34bed52e02a53edd5fa2acfc010/src/transformers/utils/chat_template_utils.py#L449
Since our models have been trained with assistant not being masked, it causes issues for downstream libraries such as MLX which always use add_generation_prompt=True, leading to incoherent and repetitive outputs
ybelkada
changed pull request status to
merged