This article is from: Kalshi Research

Translation by: Odaily Planet Daily Azuma

Editor's Note: The leading prediction market platform Kalshi announced yesterday the launch of a brand new research column, Kalshi Research, aimed at providing Kalshi's internal data to scholars and researchers interested in topics related to prediction markets. The first research report of this column has been published, titled "Kalshi Outperforms Wall Street in Predicting Inflation" (Beyond Consensus: Prediction Markets and the Forecasting of Inflation Shocks).

Below is the original content of the report, translated by Odaily Planet Daily.

Overview

Typically, a week before the release of important economic statistics, analysts and senior economists from large financial institutions provide estimates of expected values. These predictions are aggregated and referred to as "consensus expectations," which are widely regarded as an important reference for insights into market changes and position adjustments.

In this research report, we compare the performance of consensus expectations with the implied pricing of Kalshi's prediction market (hereinafter sometimes referred to as "market predictions") in forecasting the actual value of the same core macroeconomic signal — the year-over-year overall inflation rate (YOY CPI).

Key Highlights

• Overall accuracy advantage: In all market environments (including normal and shock environments), Kalshi's predictions have an average absolute error (MAE) that is 40.1% lower than consensus expectations.
• "Shock Alpha": During significant shocks (greater than 0.2 percentage points), Kalshi's predictions have an MAE that is 50% lower than consensus expectations within a one-week prediction window; if measured the day before data release, the MAE further expands to 60%. During moderate shocks (between 0.1 - 0.2 percentage points), Kalshi's predictions are also 50% lower than consensus expectations within a one-week prediction window, and this advantage increases to 56.2% the day before data release.
• Predictive Signal: When the deviation between market predictions and consensus expectations exceeds 0.1 percentage points, the probability of a shock occurring is approximately 81.2%, rising to about 82.4% the day before data release. In cases where market predictions and consensus expectations are inconsistent, market predictions are more accurate in 75% of cases.

Background

Macroeconomic forecasters face an inherent challenge: the most critical moments for predictions — when markets are disordered, policies shift, and structural breaks occur — are precisely the periods when historical models are most likely to fail. Financial market participants typically release consensus forecasts a few days before key economic data is published, aggregating expert opinions into market expectations. However, despite their value, these consensus views often share similar methodological paths and information sources.

For institutional investors, risk managers, and policymakers, the stakes of prediction accuracy are asymmetrical. During non-controversial periods, slightly better predictions provide limited value; but during market turmoil — when volatility spikes, correlations break down, or historical relationships fail — superior accuracy can yield significant alpha returns and limit drawdowns.

Therefore, understanding the behavior of parameters during periods of market volatility is crucial. We will focus on a key macroeconomic indicator — the year-over-year overall inflation rate (YOY CPI) — which is a core reference for future interest rate decisions and an important signal for measuring economic health.

We compared and evaluated the accuracy of predictions across multiple time windows before the official data release. Our core finding is that the so-called "Shock Alpha" indeed exists — that is, during tail events, market-based predictions can achieve additional predictive accuracy compared to consensus benchmarks. This outperformance is not merely of academic significance; it can significantly enhance signal quality at critical moments when prediction errors carry the highest economic costs. In this context, the truly important question is not whether prediction markets are "always right," but whether they provide a differentiated signal that is worth incorporating into traditional decision-making frameworks.

Methodology

Data

We analyzed the daily implied prediction values from prediction market traders on the Kalshi platform, covering three time points: one week before data release (aligned with the consensus expectation release), the day before release, and the morning of the release. Each market used was (or has been) a real, tradable market, reflecting actual capital positions at different levels of liquidity. For consensus expectations, we collected institutional-level YoY CPI consensus forecasts, which are typically published about a week before the official data release by the U.S. Bureau of Labor Statistics.

The sample period spans from February 2023 to mid-2025, covering over 25 monthly CPI release cycles across various macroeconomic environments.

Shock Classification

We classified events into three categories based on the "unexpected magnitude" relative to historical levels. "Shocks" are defined as the absolute difference between consensus expectations and actual published data:

• Normal events: YOY CPI prediction errors of less than 0.1 percentage points;
• Moderate shocks: YOY CPI prediction errors between 0.1 and 0.2 percentage points;
• Major shocks: YOY CPI prediction errors exceeding 0.2 percentage points.

This classification method allows us to examine whether predictive advantages exhibit systematic differences as the difficulty of prediction changes.

Performance Metrics

To assess predictive performance, we employed the following metrics:

• Mean Absolute Error (MAE): The primary accuracy metric, calculated as the average of the absolute differences between predicted and actual values.
• Win Rate: When the difference between consensus expectations and market predictions reaches or exceeds 0.1 percentage points (rounded to one decimal place), we record which prediction is closer to the final actual result.
• Prediction Time Span Analysis: We track how the accuracy of market valuations evolves from one week before release to the release date, revealing the value of continuously incorporating information.

Results: CPI Prediction Performance

Overall Accuracy Advantage

In all market environments, market-based CPI predictions have an average absolute error (MAE) that is 40.1% lower than consensus predictions. Across all time spans, the MAE of market-based CPI predictions ranges from 40.1% lower than consensus expectations (one week prior) to 42.3% lower (the day before).

Additionally, when there is a divergence between consensus expectations and market implied values, Kalshi's market-based predictions demonstrate statistically significant win rates, ranging from 75.0% one week prior to 81.2% on the release day. If we include cases where predictions are tied with consensus expectations (to one decimal place), market-based predictions are approximately 85% of the time either tied with or outperforming consensus one week prior.

Such a high directional accuracy rate indicates that when market predictions diverge from consensus expectations, this divergence itself holds significant informational value regarding "whether a shock event is likely to occur."

"Shock Alpha" Does Exist

The differences in predictive accuracy are particularly pronounced during shock events. In moderate shock events, when release times are aligned, market predictions have an MAE that is 50% lower than consensus expectations, and this advantage expands to 56.2% or more the day before data release; in major shock events, when release times are aligned, market predictions also have an MAE that is 50% lower than consensus expectations, reaching 60% or more the day before data release; while in normal environments without shocks, the performance of market predictions and consensus expectations is roughly comparable.

Although the sample size of shock events is small (which is reasonable in a world where "shocks are inherently highly unpredictable"), the overall pattern is very clear: when the predictive environment is most challenging, the information aggregation advantage of the market is most valuable.

However, the more important point is not just that Kalshi's predictions perform better during shock periods, but that the divergence between market predictions and consensus expectations may itself signal an impending shock. In cases of divergence, the win rate of market predictions relative to consensus expectations reaches 75% (within comparable time windows). Furthermore, threshold analysis further indicates that when the deviation between market and consensus exceeds 0.1 percentage points, the probability of a shock occurring is approximately 81.2%, rising to about 84.2% the day before data release.

This significant difference with practical implications suggests that prediction markets can serve not only as competitive forecasting tools alongside consensus expectations but also as a "meta-signal" regarding predictive uncertainty, transforming the divergence between market and consensus into a quantifiable early indicator for warning potential unexpected outcomes.

Derivative Discussion

An obvious question arises: Why do market predictions outperform consensus predictions during shocks? We propose three complementary mechanisms to explain this phenomenon.

Heterogeneity of Market Participants and "Wisdom of Crowds"

While traditional consensus expectations integrate views from multiple institutions, they often share similar methodological assumptions and information sources. Econometric models, Wall Street research reports, and government data releases form a highly overlapping common knowledge base.

In contrast, prediction markets aggregate positions held by participants with different information bases: including proprietary models, industry insights, alternative data sources, and experience-based intuitive judgments. This diversity of participants has a solid theoretical foundation in the "wisdom of crowds" theory, which suggests that when participants possess relevant information and their prediction errors are not fully correlated, aggregating independent predictions from diverse sources often yields better estimates.

When macro environments undergo "state switches," the value of this information diversity becomes particularly pronounced — individuals with fragmented, localized information interact in the market, allowing their information fragments to combine and form a collective signal.

Differences in Participant Incentive Structures

Consensus forecasters at the institutional level often operate within complex organizational and reputational systems that systematically deviate from the goal of "purely pursuing prediction accuracy." The occupational risks faced by professional forecasters create an asymmetric payoff structure — significant predictive errors can incur substantial reputational costs, while even highly accurate predictions, especially those achieved by significantly deviating from peer consensus, may not yield proportionate professional rewards.

This asymmetry induces "herding behavior," where forecasters tend to cluster their predictions around the consensus value, even if their private information or model outputs suggest different results. The reason is that, within the professional system, the cost of "being wrong in isolation" often outweighs the benefits of "being right in isolation."

In stark contrast, the incentive structure faced by prediction market participants aligns predictive accuracy directly with economic outcomes — accurate predictions mean profits, while incorrect predictions mean losses. In this system, reputational factors are almost nonexistent; the only cost of deviating from market consensus is economic loss, which entirely depends on whether the prediction is correct. This structure exerts stronger selective pressure on predictive accuracy — participants who can systematically identify errors in consensus predictions will continuously accumulate capital and enhance their influence in the market through larger position sizes; whereas those who mechanically follow the consensus will continue to incur losses when the consensus is proven wrong.

During periods of significant uncertainty, when the occupational costs for institutional forecasters deviating from expert consensus reach their peak, this differentiation in incentive structures is often most pronounced and economically significant.

Information Aggregation Efficiency

A noteworthy empirical fact is that even one week before data release — a time point that aligns with the typical window for consensus expectation publication — market predictions still exhibit a significant accuracy advantage. This indicates that the market advantage does not merely stem from the "speed of information acquisition" often attributed to prediction market participants.

On the contrary, market predictions may aggregate information fragments that are too dispersed, too industry-specific, or too vague to be formally incorporated into traditional econometric forecasting frameworks more efficiently. The relative advantage of prediction markets may not lie in accessing public information earlier, but rather in their ability to more effectively synthesize heterogeneous information within the same time scale — whereas consensus mechanisms based on surveys often struggle to process this information efficiently, even with the same time window.

Limitations and Considerations

Our findings require an important caveat. Since the overall sample covers only about 30 months, significant shock events are inherently rare by definition, which means that the statistical power for larger tail events remains limited. A longer time series would enhance future inferential capabilities, although the current results strongly suggest the superiority of market predictions and the differentiation of signals.

Conclusion

We have documented the significant and systematic outperformance of prediction markets relative to expert consensus expectations, particularly during shock events when predictive accuracy is most critical. Market-based CPI predictions have an overall error that is approximately 40% lower, and during periods of major structural changes, this error reduction can reach about 60%.

Based on these findings, several future research directions become particularly important: first, to investigate whether "Shock Alpha" events themselves can be predicted through volatility and predictive divergence indicators using larger sample sizes across various macroeconomic indicators; second, to determine the liquidity thresholds above which prediction markets can consistently outperform traditional forecasting methods; and third, to explore the relationship between prediction market values and those implied by high-frequency trading financial instruments.

In an environment where consensus predictions heavily rely on models with strong correlations and shared information sets, prediction markets provide an alternative information aggregation mechanism that can capture state switches earlier and process heterogeneous information more efficiently. For entities that need to make decisions in an economic environment characterized by increasing structural uncertainty and the frequency of tail events, "Shock Alpha" may not only represent a gradual improvement in predictive capability but should also become a fundamental component of their robust risk management infrastructure.

免责声明：本文章仅代表作者个人观点，不代表本平台的立场和观点。本文章仅供信息分享，不构成对任何人的任何投资建议。用户与作者之间的任何争议，与本平台无关。如网页中刊载的文章或图片涉及侵权，请提供相关的权利证明和身份证明发送邮件到support@aicoin.com，本平台相关工作人员将会进行核查。