Tencent Hybrid Puts Hy3 Down to 1 Bit, 295 Billion Parameters Can Run on Single Card According to Beating monitoring, Tencent Hybrid has released 1-bit and 4-bit compressed versions of Hy3, significantly reducing the deployment threshold for this 295 billion parameter model. The original weight of Hy3 is close to 600GB. The 1-bit version has been reduced to 85.5GiB, and a 96GB Nvidia H20 can run it. When running on a single card, users need to disable some acceleration functions and shorten the text length that the model processes at once. After using two H20s, acceleration can be turned on. The 4-bit version has a volume of 169.9GiB and requires at least two H20s. It occupies more video memory, but the effect is closer to the original model. Tencent Hybrid claims that the 4-bit version is close to the original model in terms of code, tool calls, long text understanding, and agent tasks. The 1-bit version only showed a slight decrease. After enabling generation acceleration, the speed of the 1-bit version increased by about 50%, and the speed of the 4-bit version increased by nearly 60%. Tencent Hybrid also provides a 4-bit version suitable for server deployment, which can be made into an API for multiple users or applications to call. [Original link]