ostris

/

Flex.1-alpha

like 321

Hi @ostris , congrats on releasing an amazing work! Some question:

1. Can you give more details on how to distill from 12B model to 8B?
2. Fine-tune which parameters and what losses are used.
3. Are there any plans to open source distillation code?

thanks~