GPT-6.2 Release: Can replicate PPT, build valuation models, create 3D modeling, but the hallucination rate remains as high as 59%.

CN
链捕手
Follow
6 hours ago

GPT-6.2 Release: Capable of recreating PPT, building valuation models, performing 3D modeling, but the hallucination rate still reaches 59%

By | Xiaojing

Editor | Xu Qingyang

On November 17, local time in the United States, OpenAI officially launched GPT-6.2 Astra and GPT-6.2 Sol.

This release came later than expected by the outside world. As early as the end of October, some users uncovered traces of the suspected GPT-6.2 in the usage records of ChatGPT, and thereafter model identifications and test screenshots continued to leak. Previously, media outlets had predicted that the new model would be released as soon as November 6, making the final release date 11 days later than this expectation.

OpenAI emphasized the ability of AI to take over professional tasks this time.

From recreating a complete presentation from a PPT screenshot, to extracting financial data and building valuation models, to using Blender for generating 3D models of cars and simulating the physical movement of vehicle suspensions, even utilizing CAD for designing mechanical parts and verifying assembly relationships, GPT-6.2 started to delve into tasks that previously required professionals using complex software.

OpenAI especially highlighted that the new model can reduce ineffective output and repetitive modifications, allowing the results delivered by AI to come closer to professional work standards.

In the past two months, OpenAI has successively launched models including GPT-6 Astra, Sol, and Luna, as well as GPT-6.1 Sol, Terra, and others. The newly released GPT-6.2 Astra is a flagship model upgrade following the GPT-6 Astra from September, focusing on enhancing the handling capabilities of complex professional tasks; GPT-6.2 Sol mainly strengthens the efficiency of daily work, programming, and multi-step agent tasks.

According to the latest ranking from Artificial Analysis, Astra has risen to second place globally with a score of 62, an increase of 5 points from the previous generation; Sol reached 57 points. In terms of pricing, both the input and output unit prices for Astra's API have been reduced by 20%, with cache input prices dropping by 60%; Sol maintains its original price, but the overall cost for completing certain tasks has further decreased.

It is worth noting that GPT-6.2 Astra is also OpenAI's first publicly evaluated model to meet the "High" threshold for AI autonomous improvement capabilities.

However, the impressive demonstrations and scores have also attracted different evaluations. On the release day, several developers who tested the new model weeks in advance publicly expressed their disappointment, believing that Astra may still halt prematurely during long-term software development tasks, and overly simplified outputs sometimes omit necessary information.

At the same time, as it approaches real professional work, the boundaries of model capabilities and safety controls have also become significant highlights of this release.

GPT-6.2 Release: Capable of recreating PPT, building valuation models, performing 3D modeling, but the hallucination rate still reaches 59%

OpenAI CEO Sam Altman. Image generated by AI

01 Performance and Cost

GPT-6.2 Release: Capable of recreating PPT, building valuation models, performing 3D modeling, but the hallucination rate still reaches 59%

The API price for GPT-6.2 Astra is $8 per million input Tokens and $40 per output Token, with cache input at $0.40. Compared to the previous generation, input and output prices have decreased by 20%, while cache input has dropped from $1 to $0.40, a decrease of 60%. GPT-6.2 Sol retains a price of $2 per million input Tokens and $10 per output Token.

For multi-step agents, API unit prices do not fully reflect actual expenses. When executing tasks, the model may require multiple tool calls, file readings, error handling, and retries. Even if the unit price remains unchanged, if fewer tokens are needed to complete the task, the final bill will decrease accordingly.

GPT-6.2 Release: Capable of recreating PPT, building valuation models, performing 3D modeling, but the hallucination rate still reaches 59%

In the FrontierCode 2.0 programming test by Cognition, GPT-6.2 Sol scored only 0.5 points lower than Astra, with costs approximately one-fifth of the latter. In the LifeVal assessment aimed at daily complex tasks, Sol’s score exceeded that of the previous flagship GPT-6 Astra, with per-task costs about one-seventh of the latter.

GPT-6.2 Release: Capable of recreating PPT, building valuation models, performing 3D modeling, but the hallucination rate still reaches 59%

Astra's performance in high-difficulty tasks has also improved. In the Agents' Last Exam assessment, it achieved a score improvement of 5.2 points over GPT-6 Astra, with task completion costs being less than a third of the previous generation. Compared to Claude Fable 5.2, Astra's score surpassed it by 5.5 points, with single-task costs amounting to about 5% of the latter.

02 Professional Work Delivery

GPT-6.2 Release: Capable of recreating PPT, building valuation models, performing 3D modeling, but the hallucination rate still reaches 59%

Professional work in finance, law, and consulting requires AI to process large amounts of raw data, verify information, and produce deliverables according to industry standards. OpenAI stated that GPT-6.2 Astra has shown improvements in such tasks, able to handle financial modeling, merger analysis, legal research, and data science analysis.

OpenAI's internal GDPval v2 test covers fields such as consulting, finance, and design, with Astra scoring 6.2 points higher than the previous generation, and output Tokens reduced by 22%.

In terms of legal research, in the Harvey LAB-AA v1.1 test that simulates real law firm tasks, Astra's passing rate exceeded that of Claude Fable 5.2 by more than double, with single-task costs being about one-sixth of the latter.

OpenAI's internal GDPval v2 covers work in consulting, finance, and design. Testing results showed that Astra’s score increased by 6.2 percentage points from the previous generation, with output Tokens reduced by 22%.

03 Entering the 3D World

Another capability demonstrated by GPT-6.2 Astra is direct involvement in 3D modeling and engineering design. The model is no longer just generating a segment of modeling code; it can also operate professional software to handle parts, adjust structures, and create visual content from calculations.

GPT-6.2 Release: Capable of recreating PPT, building valuation models, performing 3D modeling, but the hallucination rate still reaches 59%

GPT-6.2 Astra conducting mountain driving and chassis dynamics simulation

In the mountain driving simulation case, Astra first built a 3D model of the sports car, then wrote a Python physics solver to calculate tire ground force and dynamic changes in the suspension system, and finally imported the calculation results into Blender to generate vehicle driving animations. Through simulation, engineers can observe load transfer when the vehicle travels through complex road conditions before manufacturing physical prototypes.

The input data for the mechanical modeling case is much simpler. OpenAI only provided public photos and assembly manuals for a 1:8 scale remote-controlled off-road vehicle and an inline six-cylinder engine. Astra used this to establish 3D models of various mechanical parts and generated step-by-step assembly animations and illustrated parts catalogs.

GPT-6.2 Release: Capable of recreating PPT, building valuation models, performing 3D modeling, but the hallucination rate still reaches 59%

Additionally, one test connected CAD design with actual production. The OpenAI team provided a photo of the fan-shaped balcony of Chicago's landmark building Marina City to reference its structure for design. After multiple adjustments, Astra ultimately designed a paper art table lamp that can be printed and folded together, and created a corresponding webpage called Lamp Builder, allowing users to adjust parameters online and generate custom blueprints.

GPT-6.2 Release: Capable of recreating PPT, building valuation models, performing 3D modeling, but the hallucination rate still reaches 59%

GPT-6.2 Astra completing mechanical modeling of the remote-controlled off-road vehicle and engine

In web development, Astra created an interactive configurator for a camper van. Users can switch the body color or add a roof tent, and the page quote updates in real-time. It also generated a playable pinball table, a model of the restored Roman Colosseum, and a future city racing game called "Meridian Sound Racing" based on text descriptions.

GPT-6.2 Release: Capable of recreating PPT, building valuation models, performing 3D modeling, but the hallucination rate still reaches 59%

From Chicago's Marina City building to the paper art table lamp

The AA-3D human evaluation blind test from Artificial Analysis also compared the model's 3D generation capabilities. GPT-6.2 Astra scored 1318 Elo points, ranking first; Claude Fable 5.2 scored 1271 points, and GPT-6.2 Sol scored 1259 points.

GPT-6.2 Release: Capable of recreating PPT, building valuation models, performing 3D modeling, but the hallucination rate still reaches 59%

The interactive configurator for the camper van generated by GPT-6.2 Astra

In the AA-Robotics v0.1 preview testing physical world understanding and action planning capabilities, Astra and Sol scored 79 and 78 points, respectively, higher than Claude Fable 5.2's 75 points. However, the evaluation team reported that the top models are now capable of completing the vast majority of basic tasks in the initial tests and are increasing the difficulty of the problems.

04 Scientific Reasoning and Shortcomings

Scientific research was also a key focus of OpenAI's presentation.

In the Terminal-Bench-Science 0.1 testing, GPT-6.2 Astra scored 9 percentage points higher than the previous generation, ranking first. This test mainly assesses the model's ability to complete scientific workflow in terminal environments. In the Vals Science Index, which requires the model to interpret real biological data, design experiments, and infer systemic mechanisms, Astra also achieved a new high.

Results from several other tests were slightly inferior to Claude Fable 5.2. In the high-difficulty subset of the physical reasoning benchmark CritPt v2, factual knowledge assessment AA-Omniscience v1.1, and reasoning benchmark HLE-Diamond, Astra ranked second.

A remaining issue with the model is its tendency to provide incorrect answers when faced with uncertain facts.

GPT-6.2 Release: Capable of recreating PPT, building valuation models, performing 3D modeling, but the hallucination rate still reaches 59%

The hallucination rate test from Artificial Analysis's AA-Omniscience shows that at the highest reasoning intensity, the hallucination rates for GPT-6.2 Astra and Sol are 59% and 55%, respectively. This test primarily examines whether the model tends to guess answers when faced with factual questions that it might not accurately grasp. It does not represent the model's error rates in general Q&A, but for scenarios such as financial audits and legal citations, human verification is still required.

The programming tests did not demonstrate comprehensive improvements either. In the Artificial Analysis programming agent index, GPT-6.2 Astra’s score was consistent with the previous generation, while GPT-6.2 Sol dropped by 3 points compared to GPT-6.1 Sol.

Detailed results show that the loss of Sol mainly comes from SWE-Atlas-QnA-AA, which declined by 14 points. This test assesses the model's understanding and Q&A capabilities for complex codebases. The test results indicate that Sol is more likely to miss key facts when answering questions related to the codebase compared to the previous generation.

This is something developers working with large legacy codebases need to pay attention to. Just because the model can write code does not mean it has accurately grasped the project's history, dependencies, and specific constraints.

05 Safety Boundaries

As the model gains the ability to operate terminals, review code, and call professional software, the risks of cybersecurity have also increased.

OpenAI disclosed that both GPT-6.2 Astra and Sol have reached the cybersecurity "critical threshold" set by the company’s Preparedness Framework. Both models can more efficiently discover and leverage software vulnerabilities, as well as write safer underlying code to generate patches that retain the original business logic as much as possible.

This means the same capabilities can be utilized for both vulnerability scanning and attacking. Consequently, OpenAI has tightened restrictions on the public versions for high-risk cybersecurity tasks.

For security researchers needing to perform vulnerability testing, these restrictions will also affect some normal work. OpenAI currently grades the relevant capabilities through the Daybreak project: defense personnel can use the lower refusal rate GPT-6 Sol through Daybreak Blue; authorized red team testers can invoke the alpha version of GPT-6 Astra and GPT-6 Cyber in Daybreak Red. OpenAI stated that the GPT-6.2 series will also be introduced into this system in the future.

Additionally, OpenAI has included defense-oriented training signals in the training phase. Previously, AI had been used to find high-risk vulnerabilities like PHP2Shell. The GPT-6.2 series further enhances the ability to identify security flaws from the development source and learns to generate patches that balance security and functional integrity.

免责声明:本文章仅代表作者个人观点,不代表本平台的立场和观点。本文章仅供信息分享,不构成对任何人的任何投资建议。用户与作者之间的任何争议,与本平台无关。如网页中刊载的文章或图片涉及侵权,请提供相关的权利证明和身份证明发送邮件到[email protected],本平台相关工作人员将会进行核查。

Share To
APP

X

Telegram

Facebook

Reddit

CopyLink