On October 23, 2026 (Beijing time), OpenAI’s Engineering Blog published Part II of its scaling series. OpenAI products reach data through Habitat, its online storage platform over Azure Cosmos DB and related caches. After reliability work, this part focuses on read performance. Reads are over 90% of Habitat traffic. The layered approach: keep data near users when capacity allows; replicate only the smallest, most latency-critical datasets widely; cache the rest in Valkey close to request compute, invalidating via Cosmos change feed／CDC within seconds; and cut proxy overhead by rewriting the Habitat service in Rust.
Together those changes cut overall global read p50 from roughly 120 ms to roughly 10 ms—a 12× improvement since the performance push earlier in 2026. Slowest requests can still miss cache and fetch across distant regions. The same post walks multi-tenancy tradeoffs: noisy neighbors on shared physical partitions, hot-key throttling, and QoS. Those are engineering mitigations, not a claim that every interference is gone.
The same post says OpenAI is building a custom exabyte-scale online storage system optimized for bulky Habitat types, combining RocksDB Cloud with Raft-based replication. It is in development and implements a Cosmos-like document model with disaggregated storage—not a statement that Cosmos／Habitat has already been fully replaced. Related reading: Why Agents API Scheduled Automations Show Completed Before Work Finishes, OpenAI Tightens Isolation for Frontier Model Training, and Codex for Open Source Offers Maintainers Six Months of ChatGPT Pro.
As for Habitat latency on a given workload, whether a region can colocate storage, and how the in-house EB-scale system will feel in production—we have not tested that. Feedback can go through the site-owner contact link in the footer.