Enabling Large Models to 'Divide and Conquer' Reading and Writing: NVIDIA's TwoTower Architecture Parallelizes Two 30B Models for 2.4x Speed Boost Without Quality Loss

According to monitoring by Beating, NVIDIA has open-sourced the discrete text diffusion architecture Nemotron-Labs-TwoTower, aimed at addressing the generation speed bottleneck of large models that 'produce only one word at a time.' Traditional text diffusion models, in pursuit of parallel output, forced a single network to simultaneously handle unidirectional context understanding and bidirectional parallel error correction, resulting in a significant decline in cognitive capabilities. TwoTower adopts a decoupled dual-tower design: on one hand, it completely freezes a pre-trained autoregressive large model as a 'read-only context tower,' preserving full reasoning and commonsense capabilities; on the other hand, it independently trains a 'denoising writing tower,' which reads contextual information at the layer level through cross-attention. The writing tower employs a 'confidence-based unmasking' mechanism, prioritizing the writing of high-confidence words when predicting a block, and gradually filling in the remaining blanks, achieving parallel writing from easy to difficult. In a 30B-level hybrid architecture (Mamba-Transformer MoE) model, this design adapts using only 1/12 of the baseline model's pre-training data volume (2.1T tokens), retaining 98.7% of the quality while boosting actual generation speed by 2.42x, without adding extra VRAM caching overhead. However, since the dual towers need to reside in memory, the model's static VRAM usage has increased, and there is slight precision degradation in highly complex code and mathematical reasoning tasks. [Original Link]