[Tencent HunYuan Hy4 Open Source: 770 Billion Total Parameters, 1 Million Context Length] Beating AI News Flash: Tencent has released the Hy4 preview. The new model adopts an MoE architecture, with a total of 770B parameters and 49B activated per instance, while the context length has been expanded to 1M. Compared to Hy3's 295B total parameters, 21B activated parameters, and 256K context length, this generation has more than doubled in scale, and the long context length has been expanded by approximately four times. This time, Tencent has not only emphasized general benchmark scores but has specifically enhanced real-world tasks such as coding, office work, gaming, and scientific research. For example, the model can build a Three.js 3D website from scratch, create playable game demos in Unity, process 72 financial documents in one go, and check for duplicate reimbursements, budget overruns, and abnormal allocations. Hy4 has also been trained and iterated alongside Tencent products like WorkBuddy and CodeBuddy. A particularly unique aspect is that Hy4 has already started contributing to its own development. Tencent states that it helps optimize training methods, data strategies, evaluation systems, and underlying operators by proposing solutions, running experiments, and adjusting based on results. The code, logs, and feedback generated during experiments are fed into the next round of development, forming an initial self-improvement loop. The Hy4 preview has been integrated with WorkBuddy, CodeBuddy, Yuanbao, ima, Tencent Cloud TokenHub, and OpenRouter. Tencent had previously hinted in its financial report that Hy4 would be larger than Hy3 and would use real product feedback as a key source for model training. [Original Link]