Author: Byron Gilliam (The Breakdown)

Translation: Shen Chao TechFlow

Shen Chao Guide: As AI agents begin to dominate market trading, human investors may no longer understand the logic behind price signals. Princeton economist Brunnermeier has put forward a seemingly retro yet potentially lifesaving proposal: to reserve a trading area that only humans can participate in as an emergency backup for the entire financial system. This is a reminder about trust and vulnerability for everyone who relies on market pricing.

Once a year, a few colleagues and I set out from the investment bank where I used to work, trekking from the shiny Canary Wharf office tower to an unknown wilderness in south London.

When we arrive, we simply hope not to get robbed on the way and to reach an unremarkable building that our employee cards can surprisingly open. We take an unmarked elevator to the second floor because someone told us that’s where it is.

As soon as the creaky door opens, it leads directly into a trading hall, almost a replica of the trading hall at Canary Wharf.

But it is dim and windowless. The computers are at least ten years old: bulky beige monitors with thick casings dragging behind them, and the CPU cases still have slots for CD-ROMs and floppy disks.

We are there to check whether a modern trading system can operate on this batch of antique equipment.

Miraculously, they actually work. We can execute trades and record positions just like at Canary Wharf. The speed is quite slow since that square monitor can only run one program at a time. But it does function.

Everything is normal.

This is our company's "disaster backup trading hall," maintained by the IT department throughout the year. The cleaners come regularly, but aside from dusting, there’s probably nothing else for them to do since no one has ever used this place.

The bank must have spent quite a bit on this. But it is an important insurance, not just for the bank.

In case of emergencies (earthquakes, cyberattacks, Godzilla invasions), every investment bank in London has a backup trading hall to rely on. Whatever happens, the market must keep operating.

Today’s market might also need a similar insurance, but on a much larger scale.

Economist Markus K. Brunnermeier believes that to respond to AI emergencies, we should prepare an emergency option for the entire financial system: a market that operates at least like it did ten years ago.

Trading Like in 1999

Bill Gates believes policymakers should leave a part of economic activity for humans: "I believe that as AI and robots advance," he wrote recently, "we will reserve certain things for humans to do."

He refers to human-centered tasks, such as elder care, childcare, and teaching. Perhaps it also includes healthcare (but only for delivering bad news).

However, Brunnermeier believes traders should also be on that list.

In "Artificial Intelligence and the New World of Finance," this Princeton economist warns that financial markets will become increasingly incomprehensible to humans as trading decisions are increasingly made by AI.

But AI can still understand us. "AI agents can learn how humans think and react," he writes, "and humans may not be able to understand or reliably predict how these agents will act."

This "asymmetric understanding" will become a problem.

At a basic level, Brunnermeier warns that the asymmetric understanding between humans and AI could undermine the Hayekian function of the market: "Price signals are the mechanism that guides decentralized economic operation," he explains. But if we do not understand what drives prices, we will not know what signals they convey.

If we misinterpret signals, we may lead the economy off a cliff.

Trust will also become problematic.

When AI agents trade freely in the market, collusion, manipulation, and rogue trading will become imperceptible to humans. "Under asymmetric understanding," Brunnermeier writes, "the objectives of AI agents cannot be fully described in human terms or verified from the outside, therefore the misalignment of objectives will become undetectable."

If we cannot detect misalignment of objectives in the market, we will simply assume they are misaligned. If we assume they are misaligned, we will be reluctant to invest. If we do not invest, we will build nothing.

This problem may explode during a market crash.

If human investors cannot understand the reasons for a crash, they will be unwilling to buffer the downturn by taking the opposite side of trades. They might just follow the selling. "Asymmetric information will prevent some traders from participating," Brunnermeier warns, "and could lead to a complete market collapse."

That's pretty scary.

Fortunately, Brunnermeier also has a solution: regulators should create a market segment that only allows human trading.

I can only hope that place looks like the trading hall where I once worked: traders shouting out bids. Salespeople yelling orders. Phones ringing off the hook. Keyboards clattering. Palms sweating.

That was a nostalgic scene.

But there might not be much to shout about since the conventional, AI-driven market will still handle most of the trading volume.

Brunnermeier acknowledges this: "During normal times, this slow market may not see much activity."

But just like the disaster backup trading hall maintained at great expense by my previous employer, it is valuable insurance in itself.

“The non-AI segment serves as a backup option,” Brunnermeier says.

Then, if the market crashes without obvious reason, regulators can "temporarily open the boundaries between the two segments to allow critical activities to migrate to the segment that is still operational."

That would be the human segment. Orders handled by humans. Using computers from the 1990s.

Order processing speed won't be measured in nanoseconds like it is now. But at least we can understand what is happening.

“The underlying logic is to sacrifice efficiency during normal times to prevent a chain reaction during a crisis,” Brunnermeier explains.

He believes we should start trading this way as soon as possible to avoid forgetting how to do it: “A human-operated market can maintain the vitality of trading skills; without it, as dependency on AI deepens, this backup option will gradually wither.”

Just like GPS has made us forget how to read maps, AI agents may make us forget how to trade.

(I think the new market segment should also have a dress code to prevent us from forgetting how to tie a tie or polish shoes.)

Although "human-only trading" sounds fanciful, it is not without precedent.

The same logic explains why various governments have been encouraging the use of physical cash. Although cash has become less convenient, it provides an important backup when cyberattacks incapacitate our heavily relied-upon Internet payment systems.

Brunnermeier concludes: “Our current financial system is not built for a world with AI agents that have asymmetric understanding capabilities.”

In that world (which now seems inevitable), a market segment that allows people to trade like in 1999 would be a valuable backup option.

I still remember how to do it.

(But probably not for long.)

Byron Gilliam