Akhil Theerthala PRO

Great.
For 1, Indeed, that was a mistake on my part, I have done the validations but I forgot to add them over here. I orignally had a new section in "The Setup" which talked about using Qwen3-8b-thikning as the baseline performance. It showed ~52.8% on the validation set. I was under the impression that I replaced the section with the updated figure. I missed the fact that the figure wasn't updated. Will go back and change the figure now with the baselines of Qwen3-8B-Thinking on them.

Indeed the various other categories for the validation is an interesting idea to work on, will add that. Mainly would start by a deeper analysis of question_format Vs Validation. Then would go around and think about adding more sections like target_object category, orientation etc.

Hey, Thanks a lot for the amazing response!

I plan to expand upon this. The reason I chose this as a checkpoint is because of two key reasons, one being that for the current data scale, this was the best I could do in the budgets that I had. Secondly, I wanted to test the waters first with a smaller scale data. I trained these until the losses kept diverging to have a good estimate.

I also have to get better teacher models, as currently the chain of thought is generated by GPT-5-mini for the training set. I would like to follow OpenThoughts and use Qwen3-32B or Qwen3-Next family models to get better inference.

Additionally I want to see add more VL models like Molmo2-8B and also gemma3 12b with kimi-vl. But again that scope creep had to be controlled.

As for training on differnet datasets after each epoch, that can be done. However, that's still out of scope for the main question as the goal was to see if density approach could help or should we strictly prefer diversity.

The collection of artefacts for the blogpost can be found here

https://hf.co/collections/Akhil-Theerthala/density-vs-diversity-blogpost

The collection of artefacts for the blogpost can be found here

https://hf.co/collections/Akhil-Theerthala/density-vs-diversity-blogpost