头雁
头雁|Nov 16, 2026 08:39
While everyone was still discussing the Jensen Huang incident, NVIDIA's technical team quietly open-sourced Molt, a framework for agency reinforcement learning (RL) research. Positioned as native, minimalist, and scalable for Hugging Faces, it contains only 9.2K lines of code. It runs completely asynchronously: rollout, training, and weight synchronization can overlap, and slower episodes only affect themselves. It employs token-first contracts (token IDs, log-probs, action ranges, rewards, and multimodal tensors are fully aligned). It supports multiple algorithms: REINFORCE, GRPO, Dr.GRPO, RLOO, GAE (PPO), FlashREINFORCE, On-policy distillation, etc. -Agentic-first: The agent is the program itself. Rewards are entirely determined by your written Python code (Env or ChatAgent), supporting scorers, multi-round tool calls, VLM environments, LLM-as-judge, etc. - Hugging Face-native + PyTorch end-to-end: The model maintains a standard PyTorch module (HF layout) and is trained using NeMo AutoModel + FSDP2. The same checkpoint is used by both the vLLM and the trainer, without the need for weight transformation or bridging. - Minimalist + Scalable: Single Actor (optional KL reference) design, resulting in a clear RL graph. The same entry point, algorithm, and agent can scale from 4B to 1T-level MoEs (e.g., GLM-5.2 753B, --fsdp.ep_size 256), supporting TP/EP/CP and Adam CPU offload. https://(github.com)/NVIDIA-NeMo/labs-molt https://nvidia-nemo.((github.io))/labs-molt/ Documentation: https://nvidia-nemo.((github.io))/labs-molt/guide/ Paper: Molt: A Scalable PyTorch-Native Training Framework for Agentic Reinforcement Learning (arXiv:2607.21653)
+1
Mentioned
Share To

Timeline

HotFlash

APP

X

Telegram

Facebook

Reddit

CopyLink

Hot Reads