Written by: Deep Thinking Circle

Have you ever thought that the entire logic of how we build software might be completely overturned? For the past few decades, all software has been designed for humans. We have spent countless efforts optimizing user interfaces to make buttons easier to find, menus clearer, and workflows smoother. But what if the main users of software in the future are not humans, but AI agents? If a company has a hundred employees but a thousand AI agents working, should we still focus on optimizing human interfaces?

Recently, in an episode of the a16z podcast, Erik Torenberg, Steven Sinofsky, and Martin Casado engaged in a profoundly deep conversation with Box CEO Aaron Levie. The core question they discussed was: How will the entire software industry be restructured when AI agents become the primary users of enterprise software? This conversation made me realize that we are on the edge of a paradigm shift much more intense than most people imagine. It's not just about adding an AI feature to existing software; it's about fundamentally rethinking how software should be built, how it should interact, and how it should be used.

Software Must Be Built for AI Agents

Aaron Levie made a thought-provoking point during the conversation: If you have a hundred or even a thousand times the number of employees in AI agents, then your software must be built for agents. This is not a choice but an inevitable trend. Box is now spending as much time thinking about agent interfaces as it is about human interfaces. The speed of this transition has exceeded my expectations.

The logic behind this is quite simple. When AI agents become the primary users of software, they will interact with systems through protocols such as API, CLI (command-line interface), or MCP (Model Context Protocol). What seems to be the most effective paradigm right now? Providing access to SaaS tools for an agent that can write code, allowing it to access your knowledge workflows and contextual information. Such agents can not only read and understand information but, crucially, can also complete tasks by writing code or using APIs.

Anthropic's Claude Code, the super applications being developed by OpenAI, and the computational capabilities of Perplexity are all moving in this direction. I believe that the compound growth of this capability has only just begun. Imagine an agent that not only understands you saying "help me analyze last quarter's sales data," but can also write code to extract data, perform analysis, generate visual charts, and even proactively discover trends you might not have noticed. Where are the boundaries of this capability? I can't see them clearly right now.

But there is a key question that has been on my mind: people often say to "build things for agents," "market to agents," and "have good APIs and interface description languages." Martin Casado presented a counterpoint during the conversation that I strongly agree with: this notion is almost entirely wrong. Why? Because agents are actually best at finding the right tools and backend systems. They will not choose you just because your API documentation is well-written; they will make choices based on substantive factors like cost parameters, system reliability, data persistence, etc. What agents possess is the collective wisdom of humans using those platforms.

This perspective was an epiphany for me. As an industry, we have been too focused on interfaces and connections, overlooking the essence: we need to build better systems themselves. Agents will drive us back to the essence of technology, rather than marketing packaging. In the past, procurement decisions for enterprise software were often influenced by sales capabilities, brand impact, or even business dinners. But in an agent-led world, the weight of these factors will significantly decrease. Agents will make more rational choices based on technical merits. This presents a huge opportunity for companies that genuinely focus on technology itself.

The Threshold of Algorithmic Thinking: Not Everyone Can Command AI Agents

There was a discussion during the conversation that left a deep impression on me: the real challenges faced by non-technical personnel using AI agents. Steven Sinofsky made a sharp observation: algorithmic thinking is really, really difficult for the vast majority of employed individuals. If you ask anyone to draw a flowchart for the tasks they need to complete, they are likely to fail.

This observation hits the mark. Imagine a team of 50 marketers working on a massive product line, but only one person truly understands and can document the entire workflow. If you put these collaborative tools or AI agents in front of average employees, their ability to clearly explain "what needs to be done" is actually quite limited.

Aaron Levie's response was that this is simply a move up a level in work: you need to learn a new set of skills. This is no different from every major technological change in history. He gave an interesting example: a growth marketer at Anthropic completed work that originally required five to ten people using Claude Code. This example is significant because that person was already a systems thinker, understanding enough of the technology to achieve that.

But the key question is: if you imagine that every position has an infinite pool of engineering resources beside it, capable of automating anything that person wants to do, what will that position look like in the future? I think this is a question worth pondering. Perhaps agents will get better at guiding users towards systematic thinking, but at least at this stage, only a few people can effectively use these tools.

Steven Sinofsky shared an excellent analogy. His cousin, after graduating from an elite business school, started her first job just as the computing era was beginning. She had never used spreadsheets in graduate school and was told in her first year at work that she could hire as many interns as she wanted, so she managed an entire room of "agents"—those college students to do all the spreadsheet work. But interestingly, in the following years, she and her colleagues became spreadsheet experts. The entire abstraction layer moved upward. The work that those interns once did with calculators and HP financial calculators, she was now doing herself using spreadsheets, and she could iterate 30 times instead of just 2.

This story made me realize that we are currently at a similar stage in the development of AI agents. You might think you need 50 small agents coordinated by a super smart person. But soon, these things will fold into one another, ultimately becoming a skill set—a piece of code, we call it an agent—that understands marketing. You can ask it marketing-related questions, and then take the next step to have it execute tasks.

I believe the key turning point is: right now, you have to be a rocket scientist level of person to create 42 agents and get them all running. But this "rocket science" threshold will soon disappear, and a large block of domain expertise will return to the domain experts. This is almost identical to the evolution path of spreadsheets.

Corporate Fears: Out-of-Control Integration and Permission Nightmares

During the conversation, there was a poignant scene. Aaron Levie said that he recently expressed a similar optimistic view in front of a room full of CFOs and CIOs, and six people came over to say, "You're crazy, you've lost all credibility with me." Why? Because he claimed that integration problems would become much easier.

These enterprise IT leaders' concerns are not without reason. They fear not only the agents themselves but also the fact that humans are empowered to handle integration. When you enable employees to create new integrations, you are essentially saying, "Please come and break my core system." Imagine someone creates a new API connection between System 27 and System 38, if it's just to generate reports, then the mistake that person makes is their own business. But what if it involves writing operations?

Aaron Levie believes that for quite some time to come (N is a very large number), we will have a read-only version of agent integration. Many AI applications are currently at the consumption layer—humans are the ultimate consumers. But even at that level, enterprises are facing new challenges.

Box just launched an official CLI, and Aaron described a scenario: you provide Box CLI to Claude Code, and now you can interact with the entire Box system using natural language, orchestrating a series of operations with powerful models like Opus 4.6. This sounds cool; you can say, "Upload this entire folder on my desktop to Box," or "Process all documents in this folder," and it can do it.

But the problems that come with this are headaches. Imagine a company with 5,000 employees, each of whom has access to a shared engineering document and marketing materials repository, all using the CLI. We now face some very interesting new challenges: how to coordinate potentially 10,000 requests to the system per hour? It's not a performance issue; it's about ensuring that when one person's agent is moving files, another person's agent is not simultaneously trying to write in another folder, while a third person's agent is attempting to delete something? When these agents go wild, this will be a new problem that every CFO and CIO will have to solve, akin to putting their hair on fire.

Aaron Levie encountered this problem himself during testing. While trying to create an example of a marketing plan directory structure, he fell into some sort of loop, repeatedly creating nested directories. He joked, "I wonder what Box's deep limit on nested directories is because I'm reaching it."

This little anecdote reflects a larger issue: when agents are given execution capabilities, they may do things we did not expect. And this unpredictability is exactly what enterprises fear most.

Treating AI Agents Like Employees? It's Not That Simple

There was a discussion about how to manage AI agents that I found particularly interesting. As people start using personal agents, they will give them their own API keys, their own email addresses. So how do you prevent them from doing things they shouldn't?

Martin Casado shared a practice: give your agent your own phone number, your own credit card (hopefully a prepaid Visa card bought from CVS), and your own Gmail account. Gmail actually has many RBAC (Role-Based Access Control) permission mechanisms. You could argue that we have built many such permission systems, and we should treat the agent as an independent human.

But Aaron Levie immediately pointed out the shortcomings of this model. In a team of 50 people, do we have 100 "people" collaborating—50 humans and 50 agents, all in the same shared space? I obviously have complete supervisory authority over my agent, but what if my agent collaborates with others and inadvertently accesses resources it shouldn't? Now this autonomous, stateful agent is handling other people's information.

Here lies a fundamental contradiction. With real employees, you can't check their Slack channels, log in as them, or supervise their every move. They are responsible for their execution; in the real world, you won't be penalized for something they messed up. But with agents, you have to take full responsibility for everything they do. You need to have complete supervisory authority; they don’t have privacy rights.

So, some contradictions arise. I need to be able to give agents access, but I also need to be able to log in as them at any time, such as, "No, you messed this whole thing up, I need to roll back everything." But if I can log in as them, how can they cooperate with others in the real world while maintaining confidentiality or security of any information? Thus, agents are practically unable to exist as anything but your extension.

Aaron Levie also raised a deeper security issue: we still don't know how to keep secrets with agents. If you tell an agent "don't reveal X information in the context window," it is very challenging to solve this issue. If anything can enter the agent's context window because it has permission to access resources, then theoretically, you should assume this information could be leaked through prompt injection.

What does this mean? It means that if I know the email address of your new agent, I can email it, and I could social engineer it, which is ten times easier than social engineering a human. It is hard to have this agent access sensitive information like your acquisition documents at the same time.

I think this is one of the biggest technical obstacles facing current AI agents. Until this problem is fundamentally resolved, agents will struggle to be granted independent decision-making power and resource access rights. They will continue to exist as extensions of humans, rather than as independent entities.

The Advantage of Startups: Boldly Embracing AI Agents

There was a point in the conversation that resonated with me: the pace of diffusion of AI capabilities will be much slower than Silicon Valley folks realize. The reason behind this is that startups and big enterprises face completely different constraints.

Aaron Levie said that we see startups being able to build from scratch, free from the risks we've discussed, because they have little to lose. So we consider this as the trajectory we are in. But when you go to JPMorgan and ask them how to set up NanoClaw (a hypothetical AI agent) to automate business processes, you'll realize there is a huge gap.

Where does this gap manifest? Large enterprises have 75 legacy systems to integrate, strict compliance requirements, decades of accumulated data security standards, and complex permission management mechanisms. More critically, they have too much to lose. If something goes wrong with a startup's agent, it could just be a joke and might even become an episode in "Silicon Valley." But if a large bank's agent leaks customer data, it could be a disaster that leads to the company's collapse.

Steven Sinofsky made an excellent prediction: startups will burn through available capital, pretending that computing costs are not an issue. Many large companies will freeze out of fear, doing nothing. Then ordinary employees will start purchasing and using these tools on their own, doing what employees in large companies would do when they have money but don’t want to spend it.

In the meantime, there will be some companies willing to bet for various reasons, as they have the financial capacity to do so. These companies will become leaders in their respective fields as long as they can maintain financial health. There won’t be a situation where no one dares to enter the market because CFOs are afraid of being fired. There will be CFOs making mistakes, but that’s normal.

I believe this will create a very interesting market differentiation. Those medium-sized companies that dare to invest early and are willing to take risks may gain a competitive edge over large enterprises. They have enough resources to invest in AI but are not constrained by legacy systems and risk aversion like giants are.

At the same time, a whole new type of service companies will emerge. Imagine starting a marketing agency, engineering consulting firm, or architectural design firm from scratch, fully built on the first principles of AI agents, without information barriers and boundaries, able to provide all the context agents need to do their jobs and write software for specific needs on the fly. Such companies will have considerable disruptive potential for a period until those larger existing firms can shake off constraints.

Token Budgets: The New Battleground for Engineering Management

There was a discussion about token budgets that I found both realistic and absurd. Aaron Levie said, "The conversation around engineering cost budgets will be one of the craziest discussions in the coming years."

Why is that? Because engineering costs account for 14% to 30% of any public tech company's revenue. The difference between whether computing costs are twice that of engineering or just 3% more could be the entire EPS (earnings per share) of the company.

But Steven Sinofsky believes we still don't know the answer, and CFOs always want answers to problems they don’t know the answers to. Wall Street will force them to give a number and be accountable for it, and then they will be fired, and this loop will continue. This is nothing new; we have seen the same thing happen with every new technology regarding internet bandwidth, vacuum tubes, transistors, and the number of programmers.

However, Aaron Levie insists this time is somewhat different. He made a good point: we’ve never had a moment where every end user in an organization has the capacity to elastically provision resources on their behalf. Moreover, in many cases, the provisioning of these resources is entirely justified.

This is indeed similar to the cloud computing transition in the early 2000s, when we shifted from CapEx (capital expenditures) to OpEx (operational expenditures), and then to unlimited spending. Aaron recalled that back in the day at Box, CFOs would say, "You don’t understand, we’re an agriculture company, we only understand CapEx," or conversely, "We’re an OpEx company, we love the cloud." Accounting rule differences genuinely impacted technology adoption.

But the token budget issue is more granular. As an engineering leader, you now need to decide: Should you make engineers consider the computing budget with every prompt? Do you want long-running prompts or short ones? Do you want to parallelize? What is your tolerance for wasting tokens?

Aaron stated that his current stance is to waste a lot of tokens because that means we are trying new things. Should engineering leads be pleased that their teams are running 10 parallel experiments, even if it clearly wastes 90% of the tokens, because you may choose one path that succeeds? Or should they tell the team they need to truly design the perfect system before proceeding?

When this conversation was recorded, people were panicking about Claude Code's new Max program because they were limited after three prompts. This will be a very real topic until we can genuinely establish data center capacity.

But I agree with Steven Sinofsky's long-term perspective: this issue will eventually disappear. The primary reason is you must do the Benioff-style math. If you are paying a sales rep a million dollars a year, you have to ask how much their tools are worth. If you are paying an engineer X dollars a year, then their tools are definitively worth that investment at some point.

Moreover, the law of large numbers will resolve this issue. Eventually, you will have enough engineers using enough computing resources that things will balance out. We are currently in a transitional phase, where most people two years ago thought the spending levels on AI were just a chatbot. But they were wrong because they viewed it as a specific use case, whereas it is a platform-level shift.

The Future of SaaS Systems: The Return of Value to the Data Layer

There was a discussion about the future of enterprise systems during the conversation that left a deep impression on me. Martin Casado suggested that current SaaS providers are experiencing an interesting issue: they are not actually selling business line data; they are selling the intelligence, domain expertise, and the whole system. But agents only want to purchase data, want to authorize data and have unlimited access, but this has never been their business model.

This has always been a long-standing tension point with systems like Workday and SAP—how much API access is permitted. Salesforce underwent three massive platform redesigns for this reason. It’s a particularly interesting technical question: what does a system of record mean when people want to access data?

Steven Sinofsky plainly stated, "Trying to create a system like SAP in a vibe coding way is absurd." All domain knowledge in SAP does not just exist in a carefully organized data layer. It exists in the UI, in the middleware, and in the way you use it.

But Aaron Levie has a different view on this. He believes that if you iterate enough, agents will ultimately be largely responsible for selecting the tools they want to implement and use. Although agents cannot replace enterprise systems, after enough generations of development, agents might encounter so many obstacles with your software that they will say, "You need to ultimately retire your legacy HR system, or I can't automate this workflow for you."

This is a disruptive view. Imagine, when the number of agents is a hundred or a thousand times that of humans, if this scenario repeats itself, ultimately software stacks must be built for agents. There may be a few holding the fort, like a couple of ERP systems remaining, but for everything else, your business performance will be associated with how well your agents can access the information they need to get their jobs done. Therefore, your enterprise IT stack must be set up to support these agents effectively.

Martin Casado pointed out a nuance that I very much agree with. People often abstractly say "now you are marketing to agents," "you need to be an API," and "you need to have good interface description languages." He believes this is almost entirely wrong. What agents are really good at is finding the right backend. So they won’t say "this interface is good, this documentation is nice," they will say "the cost parameters of this, the persistence of that." They actually possess the collective wisdom of us using these platforms.

He gave an example: whenever he lets agents choose a cloud platform, the agents are using meaningful parameters rather than interface-related ones. So as an industry, we are too focused on these interfaces, thinking topics like "you need to market to agents" when in reality, we are going to be pushed to build better systems, which will be the things that get chosen.

I think this perspective is very profound. In the age of agents, technical merits will become much more important, while the importance of marketing and packaging will decline. Products that are genuinely competitive on a technical level will stand out, while those that rely primarily on sales will face challenges.

My Thoughts: We Underestimate the Scale of This Transformation

After listening to the entire conversation, my biggest takeaway is that Wall Street and the entire industry are using the wrong framework to understand the economic impact of this transformation. Aaron Levie is right; the biggest problem is that everyone is trying to figure out all the economic benefits of this, but their estimates of the scale of opportunity are at least off by an order of magnitude.

Steven Sinofsky illustrated this with historical examples. When people looked at PCs, they thought the consumption of MIPS (million instructions per second) was a finite market, not realizing what would happen if we put all that MIPS on every desktop. Moreover, people thought software just came alongside MIPS; only one person (referring to Bill Gates and Paul Allen) thought about the possibility of selling software separately.

The same thing happened in cloud computing. When people looked at the cloud, they thought we were merely moving the server business (about 60,000 a year) to other people's data centers. No one imagined that usage would grow a thousandfold.

For AI, the same thing is happening. The Wall Street model has a fixed revenue pie, with a zero-sum mindset. They believe that the amount a company spends on something each year is fixed. But when cloud computing came along, the problem Salesforce faced was that the CRM business was $2 billion a year, involving the purchase of all those servers, Oracle licenses, huge deployment pains, and years of consulting. But if you could have salespeople sign up individually, they would all register frictionlessly; that is what is happening.

I believe AI agents will lead to similar or even larger market expansions. When every knowledge worker has one or more agents working beside them, the usage of software, the amount of data processed, and the consumption of computing resources will escalate exponentially. This is not a zero-sum game; it’s not merely about transferring existing work from humans to agents, but it will create entirely new possibilities and value.

Aaron Levie mentioned that of the approximately 240 infrastructure companies he has engaged with as an investor, they have all shown asymptotic growth in the past six months. Why? Because the amount of software being written now is more than ever. With more software and more agents, there will be more consumption of computing resources. When everyone’s phones are heavily consuming AI, and mobile device AI on phones becomes a reality, the usage will increase a billionfold.

I believe we are experiencing a "transistor moment." Steven Sinofsky illustrated this with the example of vacuum tubes. There was a time when people thought the whole state of Dakota would be covered with vacuum tube warehouses, with people skating in the aisles replacing vacuum tubes just to fight World War II. Then someone said: why not use transistors instead?

Tokens might be like the MIPS of IBM back in the day. IBM sold more MIPS at lower prices each year, while still pricing mainframes based on MIPS until someone pointed out their curve was downtrending because they were producing MIPS faster than they could charge. The same thing will happen with tokens.

But in the short term, we will see enormous chaos and uncertainty. Enterprises will struggle with how much to invest, how to control costs, and how to manage risks. Startups will make bold bets and move quickly. There will be failures, and there will be successes. But in the long run, the direction is clear: software must be built for agents, APIs will become more important than UIs, the quality of systems will matter more than marketing, and computing costs will continue to decline while usage rises exponentially.

We are not experiencing a simple tool upgrade; we are witnessing a fundamental shift in computational paradigms. Companies and individuals who understand this and take action will define the tech landscape of the next decade. Those who continue to think with the old frameworks may find themselves left far behind.

This transformation has only just begun.

免责声明：本文章仅代表作者个人观点，不代表本平台的立场和观点。本文章仅供信息分享，不构成对任何人的任何投资建议。用户与作者之间的任何争议，与本平台无关。如网页中刊载的文章或图片涉及侵权，请提供相关的权利证明和身份证明发送邮件到support@aicoin.com，本平台相关工作人员将会进行核查。