Organized & Compiled: Deep Tide TechFlow

Guests: EJ and Josh, hosts of the Limitless Podcast

Podcast Source: Limitless Podcast

Original Title: If You Believe in AI, You Have to Bet on This

Broadcast Date: July 7, 2026

Key Summary

The core argument of this episode of the Limitless Podcast is straightforward: Everyone is focused on AI training, but the real next asymmetric bet is on AI inference chips. The two hosts begin with a startup called Etched, founded by two 24-year-old college dropouts, specializing in ASIC inference chips for Transformer architecture. They have raised over $800 million and signed client contracts exceeding $1 billion, with a list of investors that includes Peter Thiel, Jane Street, TSMC, and longevity enthusiast Brian Johnson.

But Etched is just the entry point. The program truly discusses the structural opportunities in the entire inference track: more than two years ago, inference accounted for one-third of chip demand, while training made up two-thirds. Now that number has flipped: inference represents two-thirds, and training one-third, with a potential tilt towards 80% inference by the end of the year. NVIDIA holds about 75% of the chip market share, but its GPUs are not optimized for inference. This is where startups have the opportunity to disrupt.

Etched's approach is not to create a better chip, but to redesign the entire rack system. They found that the actual utilization of NVIDIA GPUs on inference tasks is only 30-40%, so they optimized the whole system to achieve 80-90% utilization while reducing power consumption by 75% through voltage reduction. The downside is hardcoding: they are betting that the Transformer architecture will continue to dominate; if the model architecture changes in the future, the chip becomes obsolete. Josh calls this a life-or-death bet, while EJ believes this bet has been won for five years.

The second half of the program discusses how the public market can participate: Cerebras has already gone public, MediaTek has seen an 180% increase this year, Broadcom is designing TPUs for Google, and OpenAI's Jalapeno chip taped out in just nine months. If AI agents start operating autonomously for hours or even days, token consumption will grow exponentially, and inference chips are the necessary path.

Highlights of Key Opinions

On Inference vs. Training

• "Everyone is still focused on training, thinking NVIDIA GPUs are the ultimate answer. But inference is the real profit center. Anthropic is rumored to be profitable this quarter because of inference margins."
• "More than two years ago, inference demand accounted for about one-third; training for two-thirds. Now it has flipped, with inference at two-thirds and training at one-third, possibly leaning towards 80% inference by the end of this year. NVIDIA occupies 75% of the market share, but their chips are not optimized for inference. This shows there are no other options available in the market."
• "Chinese companies without NVIDIA GPUs have optimized inference to reach 90% of the capabilities of US top models. Inference has no moat; NVIDIA has no moat in inference."

On Etched's Technical Breakthrough

• "They are not just making a better chip. They are redesigning the entire rack system to improve inference utilization from 30-40% to 80-90%. If you spend $50,000 to $150,000 on a machine that is only using 30-40% of its true capability, you will be frustrated."
• "Power is equal to the square of voltage. They reduced voltage by half, meaning power consumption dropped by 75%. You need less power to achieve the same smart output."
• "They draw parallels with ASICs for Bitcoin mining. ASICs are specialized computers designed for specific mathematical problems, making them orders of magnitude more efficient."

On Team and Execution

• "They operate in shifts between Bangalore and the US, working 12-hour shifts each day, truly running 24 hours. They are working with TSMC, calling at midnight to test chips to see if the wafers light up green or red."
• "Google has TPUs and Amazon has Trainium, but these big companies do not sleep in factories. Etched's life and death rely entirely on this chip; if Google's TPU fails, Google remains Google."

On the Bet on Transformer Architecture

• "They hardcoded the entire computation graph into the silicon. If next year Andrej Karpathy comes up with a new architecture that is not Transformer, Etched's chip will be rendered obsolete."
• "From GPT-2 to today, Transformers have dominated for five years. OpenAI, Anthropic, Google, and Meta are all using it. There are no alternative architectures in sight in the short term."

On OpenAI's Jalapeno Chip

• "OpenAI worked with Broadcom to tape out Jalapeno in nine months. It takes nine months to have a baby, and they just had a pepper."
• "OpenAI's approach differs from Etched. They did not hardcode Transformers, but optimized the chip and rack system deeply for GPT. They have their own model, knowing what prompts users will send, which lets them optimize the entire inference path."
• "OpenAI previously had dealings with Cerebras. Cerebras just went public and saw its stock drop by 35.5%, performing poorly this year. But this isn’t an issue with the track; it’s a pricing problem."

On Investment Logic

• "Brian Johnson's original words: A few years ago, two college dropouts told me they could accelerate longevity research by making a faster AI chip. That’s their entire argument. If your chip outputs tokens faster, you can resolve research problems faster."
• "Jane Street, Peter Thiel, and TSMC's own venture capital fund have all invested. TSMC investing in you isn’t just about giving you money; they are saying, 'We want to make your chips.'
• "This is an asymmetric bet laid out before everyone. Everyone is watching memory bottlenecks and power shortages, but they forget the true source of profitability after these AI labs go public is inference."

On Nvidia and Vertical Integration

• "Don’t count NVIDIA out. They acquired Groq for $20 billion. Jensen is very aware of what is happening. Large companies can be slow to pivot, but they are moving."
• "Apple's M series chips are a model of vertical integration. If you can integrate chips, software, and models, the efficiency of your product line will change completely."

What is Inference: From Prefill to Decode

EJ: At Limitless, we've been looking for alpha, for opportunities around corners. This weekend, while everyone is setting off fireworks, we are reading about a company called Etched. This company wants to change the way we think about inference forever.

Let’s start with the basics. When you use Claude or ChatGPT, you write a prompt, hit enter, and the answer comes out. But what happens on the backend? Your prompt is sent to the servers, where there are a bunch of AI chips, usually NVIDIA GPUs. This GPU does one thing first: reads your entire prompt and processes it. This is called prefill. Then it retrieves the entire context of the conversation from memory, including all your previous prompts and the information you provided, and then it starts generating a response token by token. This is called decode. This is the whole process of inference.

Most people are still focused on training, thinking that NVIDIA GPUs are the ultimate answer. But we see a trend emerging: Google is building its own accelerators, Amazon is building its own, Cerebras has just gone public, and Groq has been acquired by NVIDIA. Etched is the most radical among them.

Josh: There is a key difference between training and inference. Training is a one-time event that usually takes months. When you hear that some company is training a new GPT or Claude model, they are talking about training. Inference is a completely different matter. And there is an interesting number: more than two years ago, inference demand accounted for about one-third; training for two-thirds. Now it has flipped, with inference at two-thirds and training at one-third, possibly leaning towards 80% inference by the end of this year.

This is quite telling, because NVIDIA occupies about 75% of the chip market share, but their GPUs are not optimized for inference. NVIDIA's market share is rising while inference demand is also increasing, but their chips are not good at it. This can only prove one thing: there are no other usable options in the market. No one really figured out how to produce custom inference chips at scale. So the entire market has to rely on NVIDIA. But companies like Etched are emerging to try to fill this gap.

EJ: You could even say that Chinese companies have proven this path is viable. They don’t have NVIDIA GPUs, but they can train models that reach 90% of the capability of American frontier models. They rely on extreme innovation in inference. So inference is the next true battleground, and in this battleground, NVIDIA has no moat.

Etched's Two 24-Year-Old Founders

EJ: Etched was founded by two 24-year-olds. They started three years ago, betting not on creating a general-purpose GPU, but on making a specialized ASIC chip entirely for Transformer architecture. Now they have raised over $800 million and have client contracts exceeding $1 billion. In early tests, their server racks have already achieved cutting-edge levels in latency, power consumption, and inference workload.

You might wonder: this is a private company; how can I invest? Indeed, you can’t invest directly. But the public market has paths to gain exposure to this sector. We will discuss this further. First, let’s talk about why this company managed to reach such a high valuation even before officially releasing products.

The answer is simple: they are not just making a chip. They are building an entire rack system. That is their core argument. They observed how inference actually works, looked at NVIDIA GPU's performance, and found that the true utilization of GPUs in inference is only 30-40%. If you spend $50,000 to $150,000 on a machine that is only using 30% of its true capability, you'll be very frustrated. So they not only designed a better chip but a whole system that can be placed in data centers, increasing inference utilization to 80-90%.

They mainly did two things. The first thing is they figured out how to achieve the same intelligent output with lower voltage. They redesigned the entire chip around the Transformer architecture. Power is equal to the square of voltage; by halving the voltage, they reduced power consumption by 75%. At the data center level, this means you can save tens of millions of dollars in electricity and cooling costs.

The second thing is that because the entire system is designed for inference rather than training, they optimized the entire process from prefill to decode. NVIDIA GPUs are generalized computing devices that can handle training, inference, and graphic rendering. Etched does one thing but does it to the extreme.

Josh: Here is a particularly fierce analogy. They drew inspiration directly from ASICs for Bitcoin mining. Bitcoin miners are specialized computers designed for specific mathematical problems, achieving efficiencies several orders of magnitude higher. What Etched is doing is essentially this logic: creating a dedicated chip that addresses the specific mathematical problems of Transformer inference.

Teams Sleeping in TSMC Factories

EJ: What impresses about this company is not just the technology, but the execution method. They are already collaborating with TSMC. They convinced TSMC that their technology is good enough to warrant making a tape out. Half the team is in Bangalore and half in the U.S., working in shifts of 12 hours each day, operating truly around the clock. They have the opportunity to test chips at TSMC. They call the factory at midnight to check if the chips on the wafers light up green or red. This work is tough enough to make people quit.

This is interesting when compared to Google and Amazon. Google has TPUs, and Amazon has Trainium, but these big companies do not let employees sleep in factories. For them, if the chip project fails, the company still runs. Etched is different; its survival depends entirely on this chip. Therefore, when they are hiring, they are looking for people excited about working in a company whose success hinges on this chip.

Josh: This is also why they have been able to achieve this in three years. The normal process takes one and a half to two years, but they managed to compress it.

What the Investors Are Betting On

EJ: Look at their list of investors, and you'll understand what problems these people are trying to solve. Brian Johnson, the guy obsessed with longevity who tracks a hundred metrics daily, why is he on the cap table? Brian Johnson tweeted: "A few years ago, two college dropouts told me they could accelerate longevity research by creating a faster AI chip." This is his entire logic. If your chip can spit out tokens faster, you can solve research problems more quickly. Finding drugs, protein folding, disease mechanisms—all need AI inference. Who is quicker wins.

Jane Street is in there as well, one of the world's best quantitative hedge funds. Peter Thiel. TSMC also invested through its venture capital fund. When TSMC invests in you, they’re not just giving you money; they are also saying, "We want to manufacture your chips." This signal is extremely strong.

Josh: Longevity, finance, semiconductor manufacturing—people from these totally unrelated fields are putting money into the same company, showing they see not just a small opportunity but that improving fundamental computation efficiency will change all AI-dependent industries.

The Life-or-Death Bet on Transformer Architecture

Josh: However, there is a boundary condition I had not fully recognized before. They have made a very specific bet, betting on the Transformer architecture. From GPT-2 to today, all cutting-edge models run on Transformer. Its core mechanism is next token prediction, learning recursively through latent space. But Etched’s chips are hardcoded for this architecture. If the architecture changes, such as if someone comes up with something better than Transformer, these chips would require a significant portion of the tech stack to be rebuilt. This poses a life-or-death risk; is it an acceptable wager?

EJ: You are correct. They have hardcoded the computation graph into the silicon. If next year Andrej Karpathy introduces a completely new architecture that is not Transformer, Etched's chips would become useless. But the upside is also evident: if you bet right, the efficiency gains could be 10x to 50x. From GPT-2 to now, Transformers have dominated for five years. OpenAI, Anthropic, Google, and Meta use it. In the short term, there are no alternative architectures in sight. So while this bet is extreme, the probability isn't crazy.

EJ: Compare this to OpenAI's own Jalapeno chip. OpenAI did not hardcode Transformer but deeply optimized the chip and rack system for GPT. Why can they do this? Because they have their own models. They know what prompts users will send and how to load tokens. They have vertically integrated chips, models, and distribution. This is actually safer than Etched's purely third-party approach. To achieve the same level of optimization, Etched would either need to be acquired by a Frontier lab or serve multiple clients but could not optimize to perfection for a single model.

Josh: And OpenAI, in collaboration with Broadcom, taped out Jalapeno in just nine months. Nine months, just like having a baby. They’ve just had a pepper.

EJ: OpenAI is not new to this. They previously had dealings with Cerebras. Cerebras just went public and saw its stock fall by 35.5%, performing poorly this year. But this isn’t an issue with the track; it’s a pricing problem. Many people in the market believe Cerebras has been unjustly punished.

Inference is the New Moat

EJ: The reason we are doing this episode is that many people don’t realize: inference has become the new moat for training better models, as well as the new moat for sending tokens to a large number of users.

Most people are still at the stage of thinking, "Using LLM is like using Google." There are probably less than 1% of people on Earth who have really run an autonomous agent for more than an hour. But the trend is clear: you will soon have a multitude of AI models autonomously working for you for hours or even days. This will burn vast amounts of tokens. You want them to spit out as many tokens as possible in a unit of time because you will get answers faster, complete work faster, and defeat competitors faster.

To achieve this, you need a completely different chip architecture. And NVIDIA, the big brother of all GPU architecture companies, has not yet solved the problem of inference efficiency.

Anthropic is rumored to become profitable this quarter thanks to inference margins. Everyone is looking at training costs, but the real source of profits after these AI labs go public is inference. Training is upfront investment; inference is continuous cash flow.

Paths in the Public Market

EJ: You might say, Etched is a private company; I can’t invest. But there are paths in the public market. Cerebras has gone public. MediaTek is helping design these dedicated chips and has risen 180% this year. Broadcom is designing TPUs for Google and rose about 10% today. Groq was acquired by NVIDIA for a rumored $20 billion. NVIDIA is not unaware of this trend; they are just pivoting.

Josh: If Apple’s M series chips are a model, you know that vertical integration can completely redo a product line. Now OpenAI is making Jalapeno, Google is making TPUs, and Amazon is making Trainium; everyone is doing ASICs. This trend has just begun.

EJ: This is an asymmetric bet laid out before everyone. Everyone is looking at memory bottlenecks, which is crucial. Everyone is looking at power shortages, which is also crucial. But many people forget that when these AI labs go public, the real source of profitability is inference. If you believe in agents and the future of autonomous work, you must bet on inference chips. This is not optional.