Coinbase Reveals How One Configuration Error Triggered a 50-Minute Outage

CN
14 hours ago

Key Takeaways

  • Coinbase traced the 50-minute outage to an undetected Kubernetes resource name collision during a routine infrastructure update.
  • The incident disrupted transfers, Coinbase Card debit purchases, institutional settlements, and onchain services across multiple platforms.
  • The outage also disabled Coinbase’s normal rollback process, prompting new safeguards and infrastructure changes.

Crypto exchange Coinbase (Nasdaq: COIN) has identified the software error behind last week’s outage that left customers unable to complete transfers, make purchases with Coinbase Card, and access several onchain services for about 50 minutes.

According to a postmortem published on July 21, engineers were deploying what appeared to be a routine, low-risk update when an undetected Kubernetes resource name collision caused the update to modify the wrong network component.

Coinbase said:

“This was caused by an unintended misconfiguration of an important network component as a result of a routine configuration update. At no point were customer funds at risk.”

Kubernetes is software Coinbase uses to organize and run applications across its cloud infrastructure. The resource name collision caused the update to alter part of the company’s Istio ingress gateway instead of only the intended component. The gateway acts like a traffic controller, directing requests between Coinbase’s internal services.

Once the gateway went offline, systems responsible for processing trades, transfers, settlements and payment authorizations could no longer communicate.

The disruption affected Coinbase’s retail, institutional and developer platforms from approximately 12:37 p.m. ET to 1:25 p.m. ET.

Some retail customers could not complete deposits, withdrawals or off-platform transactions. Transfers already underway appeared stuck but were processed after service returned. Coinbase Card debit purchases were declined, while credit card purchases continued.

The outage also interrupted onchain swaps through Coinbase DEX on Base and Solana. Coinbase Exchange and Prime customers experienced delayed or failed transfers and settlements, while developer platform customers could not onboard users, move funds, or use onramp services.

The impact highlights the growing importance of reliability as Coinbase expands beyond spot crypto trading. Its planned “everything exchange” products, including tokenized stocks and perpetual futures, are widening the range of financial activity dependent on the company’s infrastructure.

Coinbase has separately described plans to make the platform a broader financial hub, with new trading, payments and money-management services spanning stocks, derivatives, stablecoin payments and developer tools. That expansion raises the stakes when shared infrastructure fails.

The gateway was restored at 1:20 p.m. ET, and Coinbase declared the incident mitigated three minutes later. Remaining transaction backlogs cleared over the following hours.

Recovery took longer because Coinbase’s deployment tools relied on the failed gateway. Engineers could not use the standard rollback process and instead manually launched an emergency deployment through the company’s cloud provider using temporary privileged access.

Coinbase is now adding checks to block naming conflicts, separating recovery tools from the infrastructure they manage and testing emergency access procedures more regularly.

Coinbase emphasized:

“Our goal is always zero downtime. We are actively redesigning our infrastructure to ensure that, in the highly unlikely event of another failure like this one, we can swiftly revert and recover.”

免责声明:本文章仅代表作者个人观点,不代表本平台的立场和观点。本文章仅供信息分享,不构成对任何人的任何投资建议。用户与作者之间的任何争议,与本平台无关。如网页中刊载的文章或图片涉及侵权,请提供相关的权利证明和身份证明发送邮件到support@aicoin.com,本平台相关工作人员将会进行核查。

Share To
APP

X

Telegram

Facebook

Reddit

CopyLink