[Meituan Open-Sources the First Trillion-Parameter Large Model LongCat-2.0 Using Domestic Chips for Both Training and Inference] According to monitoring by Beating, Meituan has open-sourced the ultra-large-scale mixture of experts (MoE) model LongCat-2.0. The model boasts a total of 1.6 trillion parameters, with approximately 48 billion activated parameters per token, and supports a 1M ultra-long context. This is the industry's first trillion-parameter large model to complete the entire training and inference process relying solely on domestic computing power. It underwent pre-training on over 50,000 domestic AI chip clusters, processing 35 trillion tokens, successfully validating the engineering stability of domestic computing power in supporting cutting-edge large models. The core updates of LongCat-2.0 focus on long-context handling and inference efficiency. LongCat Sparse Attention (LSA) addresses memory read and computation overhead caused by sparse attention indexing by introducing flow-aware indexing, cross-layer indexing, and hierarchical indexing. These enhancements make index reading during long-text inference more continuous and allow partial reuse of index results between adjacent layers. The model also integrates a 135-billion-parameter 5-gram embedding module, which expands the embedding space by modeling adjacent token combinations, thereby enhancing local context representation. Compared to relying solely on MoE expert routing, this type of pre-embedding can reduce some memory read/write pressure during large-batch inference. In mainstream evaluations such as SWE-bench Pro and code assessments for agents, LongCat-2.0's performance approaches or even surpasses some mainstream closed-source models. [Original Link]