
Hong Kong Research and Development Center Collaborates with Dolby Laboratories
StereoGen Technology Breakthrough in Text-to-Stereo Generation
Wins ACM MM Best Paper Award
The Hong Kong Generative Artificial Intelligence Research Center (HKGAI) has achieved another milestone in scientific research. A spatial audio research paper jointly completed by Professor Xue Wei's team at HKGAI and the international audio technology company Dolby Laboratories, titled "Discrete Coding and Masked Modeling for Text-to-Stereo Audio Generation," focuses on Text-to-Stereo Audio Generation technology, aiming for artificial intelligence to automatically generate high-quality stereo audio with spatial orientation and movement trajectories based on text descriptions.
Currently, many AI voice generation tools on the market primarily output monophonic or insufficiently spatial audio, making it difficult to realistically present the direction and distance of sound sources. The research team points out that humans can determine the position of sound because there are subtle time and volume differences in sound received by the left and right ears, and StereoGen is based on this principle to achieve innovative breakthroughs, allowing AI to possess spatial auditory understanding capabilities closer to those of humans. The StereoGen system consists of three core technical components.

StereoCodec: Responsible for converting stereo audio into high-fidelity discrete coding, preserving important spatial information from both channels.
StereoGen: Uses a hybrid Transformer architecture and masked generative modeling technique to generate stereo sound effects with realistic movement trajectories based on text instructions such as "a motorcycle speeds from right to left."
StereoCLAP: Serves as an evaluation framework to assess the degree of match between generated sound and text descriptions in terms of semantics and spatial positioning, utilizing innovative directional difficulty negative sample mining techniques.
The research team states that this technological breakthrough is expected to bring significant changes to AI film production, animation creation, and game development. In the future, creators will only need to input script content or scene descriptions, and the system can automatically generate environmental sound effects that align with situational and directional changes, significantly reducing post-production sound costs and technical barriers.
More importantly, the spatial audio foundational capabilities established by StereoGen will become an essential component of future multimodal World Models. By combining sound, image, and language understanding abilities, artificial intelligence is expected to better simulate and understand real-world environments, driving the development of the next generation of immersive digital content experiences.
Founded in 2023, HKGAI is an artificial intelligence research institution established with funding from the Hong Kong government's AIR@InnoHK innovation platform, led by the Hong Kong University of Science and Technology, dedicated to developing locally self-research generative artificial intelligence foundational models and ecosystems. Winning the ACM MM Best Paper Award not only highlights the international competitiveness of Hong Kong's research team but also reflects that local AI foundational research results are gradually exerting influence on the global academic stage. This was honored with the highest recognition, the "Best Paper Award," at the international top multimedia academic conference ACM Multimedia 2026 (ACM MM 2026).
This article is reproduced from the Hong Kong Economic Journal, and the copyright belongs to the original author and original media.
免责声明:本文章仅代表作者个人观点,不代表本平台的立场和观点。本文章仅供信息分享,不构成对任何人的任何投资建议。用户与作者之间的任何争议,与本平台无关。如网页中刊载的文章或图片涉及侵权,请提供相关的权利证明和身份证明发送邮件到[email protected],本平台相关工作人员将会进行核查。