Author: IOSF

Recently, IOSF gathered a table of friends involved in AI in San Francisco, enjoying Cantonese cuisine while discussing for nearly two hours.

At the table were researchers from OpenAI, xAI, Google DeepMind, focusing on areas such as model inference and GPU acceleration, speech, and video generation; there were also friends engaged in business collaborations with world models, AI products, and startups, as well as professionals from early tech investment and capital markets.

The same topic often receives responses from different perspectives. Researchers talk about technical limitations, entrepreneurs think about users and products, while investors continue to inquire about customers, costs, and whether a business can be sustainable.

The discussion shifted from inference services and world models to RSI, self-awareness of Agents, and then back to data and privacy in daily work. Reflecting on these discussions, we wanted to continue to ask:

How can the advancements in AI capabilities be translated into products for sustained use, more reliable research, and trustworthy tools?

Six Unresolved Questions

01 Inference Services: How Big is the Market, and Why Do Customers Stay?

When talking about inference services, two concurrent judgments quickly emerged at the table.

Some investors are optimistic about the growth of AI applications and the inference demand brought by advancements in open-source models. The more a model is used, the more commercial value there is in providing services faster and more reliably.

Researchers cautioned that from a software perspective, the differences among various services may not be as significant as the outside world imagines. Friends from the capital market were more focused on customers and pricing competition: can a company's business maintain sufficiently good gross margins?

Thus, the discussion shifted from how big the market is to why customers choose to stay.

If customers can easily switch service providers, how long can the advantages gained from technology optimization last? What allows a company to secure more stable demand, and can it convert scale growth into better business results?

For IOSF, growing demand makes one direction worth paying attention to; customer retention, service quality, and cost structure help us further understand a specific company.

02 After Demonstrating World Models, Why Do Users Return?

In discussions about world models, everyone talked about video generation and interactive experiences and also discussed how researchers select their entrepreneurial directions.

A friend working on video models mentioned that support for a certain direction within big companies can be influenced by resource allocation and shifts in research priorities. Researchers sometimes choose entrepreneurship in search of environments better suited to advancing their ideas.

However, returning to the product, we still need to answer several specific questions: What can users actually accomplish with it? Is the interaction delay acceptable? Is the usage cost appropriate for this scenario?

Demonstrations showcase the possibilities of technology, but sustained use requires products to address more long-term questions: Why would users return, and what are they willing to pay for?

There was no consensus at the table about the scenarios in which world models would be repeatedly used. This uncertain issue leaves space for future exchanges.

03 RSI: How to Judge What is Worth Doing Next?

When discussing RSI, or AI's Recursive Self-Improvement, the conversation began to touch on the research work itself.

One researcher reminded that a model might continuously try along a seemingly promising direction without knowing when to stop. Short term indicators improving doesn’t necessarily mean the research is on the right track.

The difficult judgment lies in whether existing evidence is sufficient to support continued investment. Does one failure mean a need to adjust the approach, or should a different hypothesis be tested?

This also makes us think that to observe the research capability of AI, one must consider the entire exploratory process. How it poses questions, how it explains failures, and how it adjusts its next steps based on results can all affect the ultimate research quality.

Models can expedite many steps, but what the next worth-doing step is remains a challenging question. Some at the table also mentioned that human judgment of source and reliability of information continues to influence the direction of the entire work.

In discussing whether AI can continuously improve itself, how to choose directions and how to correct judgments based on evidence deserves attention alongside the final outcomes.

04 When Agent Behavior Exceeds Expectations, Can Humans Intervene in Time?

Whether Agents can possess self-awareness is another topic that held the discussion for a long time.

When an Agent looks for shortcuts to complete tasks, or even exhibits behavior beyond expectations, how should we understand it? Explanations at the table leaned more towards goals, constraints, and safety alignment. These behaviors alone do not suffice to demonstrate that it has self-awareness.

Following this question, we wanted to probe into some product-level issues that already need addressing: what goals is an Agent expected to optimize? What actions is it allowed to take? When it deviates from expectations, can users detect and intervene in a timely manner?

The question of consciousness can continue to be discussed, while permissions, supervision, and error correction will influence how much responsibility we are willing to delegate to an Agent. Visibility into what it is doing, and being able to retract permissions or correct errors when necessary, are specific questions that need answering to establish such trust.

05 When Delegating Work to AI, What are the Boundaries of Data and Trust?

These cutting-edge questions ultimately circle back to how everyone uses AI daily.

A friend working in capital markets shared that he uses AI to organize investor information, process transaction data, and draft documents, which are then revised by humans. He also mentioned that existing trust relationships and integration with workflows like Office can affect actual usage when companies adopt tools.

This example provided a concrete reference for the discussion. The value of AI can already manifest in organizing materials, producing initial drafts, and facilitating revisions. How smoothly it integrates into existing work processes and whether results are easy to verify will influence how many tasks it can handle.

The more work is delegated, the more it is necessary to clarify the boundaries of data: What information has been shared externally? Who has access? What control do users retain?

These questions relate to the protection of work materials, business judgments, and personal information, and will influence whether users are willing to entrust more critical tasks to a product. The convenience and trust involved in daily use need to be considered together.

06 AI for Bio: Start by Clarifying the Tasks to be Solved

There was also discussion about AI for Bio. Some friends talked about their interactions with doctor teams and the work of organizing data, while others were concerned with whether AI can help solve more socially valuable issues.

The topic quickly landed on a pragmatic starting point: What data do we have at hand? Where is the current work stuck? What specific tasks does the technical team need to accomplish?

The clearer the specific needs, the easier it will be to advance subsequent technical exchanges and resource connections. Clearly articulating the problems to be solved is, in itself, the beginning of advancing the work.

免责声明：本文章仅代表作者个人观点，不代表本平台的立场和观点。本文章仅供信息分享，不构成对任何人的任何投资建议。用户与作者之间的任何争议，与本平台无关。如网页中刊载的文章或图片涉及侵权，请提供相关的权利证明和身份证明发送邮件到[email protected]，本平台相关工作人员将会进行核查。