CLIMB: Clustering-based Iterative Data Mixture Bootstrapping for Language Model Pre-training

Shizhe Diao, Yu Yang, Yonggan Fu, Xin Dong, Dan Su, Markus Kliegl, Zijia Chen, Peter Belcak, Yoshi Suhara, Hongxu (Danny) Yin, Mostofa Patwary, Yingyan Celine Lin, Jan Kautz, Pavlo Molchanov

December 2025

arXiv

Type

Conference paper

Publication

Advances in Neural Information Processing Systems (NeurIPS)