Farewell to the "seesaw effect" of big model reinforcement learning, Xiaomi and Peking University propose the MOPD distillation framework According to Beating monitoring, large models often face the "seesaw effect" when using reinforcement learning (RL) to enhance domain specific abilities during the post training stage: when integrating multiple abilities such as mathematics, code, and instruction following, they interfere with each other, resulting in the final model performance being inferior to that of proprietary experts in various fields. Peking University and Xiaomi Model Team Jointly Publish Paper, Proposing Multi Mentor Online Distillation Framework MOPD（Multi-Teacher On-Policy Distillation）。 This method first allows RL experts from different fields (such as mathematics and code) to act as "mentors" for parallel and independent reinforcement learning, and then distills these mentors into the same student model in the policy space. Specifically, the student model samples on its own generated trajectory and minimizes KL divergence at the Token level with the corresponding domain mentor. This brings multiple advantages: it eliminates exposure bias in traditional offline fine-tuning through online (On policy) sampling, significantly reduces training variance and improves sample efficiency through token level intensive supervision, and allows for parallel research and hyperparameter adjustment by mentors in various fields. Experiments on the Qwen3-30B-A3B model showed that MOPD improved the normalized composite score by 5.5 points (reaching 0.937) compared to baselines such as Joint Reinforcement Learning (Mix RL), almost fully preserving the vertical abilities of each mentor. This scheme has been successfully deployed in the post training of Xiaomi's industrial scale model MiMo-V2-Flash (309B). The study also pointed out that the stability of distillation is highly dependent on "homologous mentors", meaning that mentors and students must start from the same SFT checkpoint. If external strong models with large distribution differences (high KL divergence) are forcibly introduced (such as using the 235B model to distill 30B students), it will actually lead to the collapse of the optimization process. [Original link]