Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

JD.com Consolidates Lakehouse Workloads on Apache Doris

AI summary

JD.com is gradually migrating workloads from Presto and StarRocks to a unified Apache Doris lakehouse engine, using Multi-Catalog, the Nereids optimizer, and vectorized execution. The company also changed scan-task scheduling from pull to push and improved partition pruning, file listing, and execution efficiency. JD.com claims the platform handles a million queries and PB-scale data scans daily, with Agent queries accounting for about 20% of the workload.

In-depth insight

The production migration offers a concrete example of consolidating lakehouse engines while adapting Apache Doris for BI, data-quality, and Agent query workloads.

在 Doris Summit 2026 上,蔡聪辉(京东零售 OLAP 引擎 Tech Lead、Apache Doris PMC Member)分享了 Apache Doris 湖仓查询底座的优化实践,本文整理自该演讲。


随着业务分析需求持续增长,以及 AI Agent 带来新的查询负载,京东原有多引擎并行的湖仓分析架构面临查询场景复杂、资源调度难度提升与统一治理等挑战。

为构建更加统一、高效的湖仓分析底座,京东逐步将 Presto、StarRocks 等分析引擎承载的业务向 Apache Doris 收敛,并依托 Doris 的 Multi-Catalog、Nereids 优化器和向量化执行引擎,围绕查询调度、分区裁剪、文件扫描及执行资源管理开展全链路优化。

目前,京东基于 Apache Doris 构建的统一湖仓分析引擎已支撑日均百万级查询与 PB 级数据扫描,Agent 智能体查询占比约 20%,承载 BI 报表、数据质量任务与 AI Agent 等多种分析负载。

本文将结合京东的实际生产场景,深入解析其统一湖仓架构演进,以及面向大规模湖仓查询的核心技术优化实践,为大型企业建设高性能、可扩展的湖仓分析平台提供参考。

#JD.com#Apache Doris#Presto#StarRocks#Apache Doris Multi-Catalog#Apache Doris Nereids#Hive#Iceberg#Hudi#Paimon#Apache Doris BE

Original · 12 min readRead the original
Group chat

No comments yet. Start the conversation.