Tencent Open-Sources Hunyuan Hy4 Preview: A Leap in Scale and Capability
Tencent has released the preview version of its latest large language model, Hunyuan Hy4, as an open-source project. This iteration marks a substantial leap in both architectural scale and practical functionality, pushing the boundaries of what domestic AI models can achieve.
Architecture & Scale: Exponential Growth in Parameters and Context
Hy4 is built on a Mixture of Experts (MoE) architecture. It boasts a total parameter count of 770 billion, with approximately 49 billion parameters activated per inference. A standout feature is its massively expanded context window of 1 million tokens, up from 256K in the previous Hy3 generation.
- Total Parameters: 770B
- Activated Parameters: ~49B
- Context Window: 1M tokens
This represents a more than twofold increase in total parameter scale and a roughly fourfold expansion in long-context handling capability compared to Hy3, enabling the processing of vast amounts of information in a single instance.
Focus on Real-World Tasks: From Benchmarks to Practicality
Moving beyond generic benchmark scores, Tencent highlights Hy4's proficiency in tackling complex, real-world challenges. The model demonstrates advanced capabilities in creative and analytical domains.
For instance, Hy4 can build an interactive 3D website from scratch using Three.js based on natural language instructions. In game development, it can generate a playable game demo within the Unity engine. For enterprise tasks, the model can process up to 72 financial documents simultaneously to identify duplicate reimbursements, policy violations, and budget anomalies.
This practical prowess is partly attributed to its co-training and iterative development alongside Tencent's own products like WorkBuddy and CodeBuddy. Learning from real user feedback ensures the model's outputs align closely with actual workflows.
Evolving R&D: The Emergence of a Self-Improvement Loop
A particularly distinctive aspect of the Hy4 project is the model's emerging role in its own research and development. Tencent reports that Hy4 assists in optimizing training methodologies, data strategies, evaluation frameworks, and even low-level computational operators.
The process involves the model proposing optimization plans, running experiments, and refining strategies based on outcomes. The code, logs, and feedback from these experiments are then fed back into the next R&D cycle, creating a preliminary self-improvement loop. This points toward a future with greater automation in AI development.
Ecosystem and Availability
The Hy4 preview is now accessible through several Tencent platforms including WorkBuddy, CodeBuddy, and Tencent Cloud's TokenHub, as well as the third-party platform OpenRouter. Tencent had previously indicated in its financial reports that Hy4 would be significantly larger than its predecessor and that real-world product feedback is a crucial component of its training regimen.