Can open-source models running locally reach the current level of OpenAI Astra in just 2 months? According to *The Information*, insiders revealed that Astra uses limited recurrent depth (recurrent depth / looped transformer)—before generating the next token, the same set of layers loops through the internal state multiple times. The report describes this as a technique to improve performance while reducing visible chain-of-thought reasoning. If this is true, it’s great news for open-source models, because compared to large models, recurrent depth provides a bigger performance boost for smaller models. Recurrent depth allows some capabilities that would normally require larger parameters to be achieved through more computation during inference instead. The trade-off is higher inference compute costs. More importantly, recurrent depth and MoE (Mixture of Experts) architectures have clear structural synergy, making them naturally better suited for MoE architectures than dense Transformers. With MoE, each loop can call different experts, meaning the same shared layers can perform different computations in different loops. Previous research papers have already shown that recurrent depth techniques can enable a 3.5B model to achieve scores comparable to a 50B model. While it’s impossible to confirm that open-source small models will definitely reach Astra’s level within 2 months, architecturally, they clearly have greater advantages. Moving forward, models at Astra’s level might be caught up to by open-source small models at a faster pace than previous cutting-edge models. So... Are GPUs about to get even more expensive again?