惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

A
About on SuperTechFans
Y
Y Combinator Blog
钛媒体:引领未来商业与生活新知
钛媒体:引领未来商业与生活新知
Microsoft Security Blog
Microsoft Security Blog
aimingoo的专栏
aimingoo的专栏
I
InfoQ
C
Check Point Blog
IT之家
IT之家
MyScale Blog
MyScale Blog
Apple Machine Learning Research
Apple Machine Learning Research
Vercel News
Vercel News
Last Week in AI
Last Week in AI
GbyAI
GbyAI
P
Proofpoint News Feed
量子位
Stack Overflow Blog
Stack Overflow Blog
Microsoft Azure Blog
Microsoft Azure Blog
月光博客
月光博客
阮一峰的网络日志
阮一峰的网络日志
人人都是产品经理
人人都是产品经理
B
Blog
T
The Blog of Author Tim Ferriss
H
Help Net Security
云风的 BLOG
云风的 BLOG

ClickHouse

clickhouse 统一查询语句,凌晨查询结果跟白天不一致 - V2EX 有没有熟悉 clickhouse 的? clickhouse 对于分布式支持的如何? ClickHouse 的 MaterializedMySQL 引擎 两条数据库创建语句产生了同样的效果 CREATE DATABASE hello1; 与 CREATE DATABASE hello ON CLUSTER 'xxxxx'; 大佬们,我又来了!群晖装 clickhouse,撑得住吗? 究竟是什么在占用着内存 求大佬优化 3000w 数据多 UNION clickhouse 文档里的划分冷热多盘存储配置真的是按时间划分冷热数据的吗? 请教各位大佬关于 clickhouse 的问题 我这个场景, clickhouse 适用吗?
求大佬优化一下 3000 万数据的 NOT IN 查询
sunrealzhang · 2023-12-13 · via ClickHouse

我有一个 3000w 行的数据表,我需要在这个表上统计从某一年开始新参保的人数,原数据库是 ORACLE ,用的是

AND A.AAC001 NOT EXISTS
(SELECT 1
FROM AC02_TEMP AS B WHERE A.AAC001 = B.AAC001 AND
B.AAC030 < '2018-01-01 00:00:00')

的语法,在 clickhouse 上我试了 LEFT JOIN 和 NOT IN ,性能均不理想

SELECT COUNT(1)  AS "新参保人数"
FROM AC02_TEMP AS A
WHERE A.AAB301 IN (SELECT AAB301 FROM AA26 WHERE AAA148 = '130800')
AND A.AAE200 = '41'
AND A.AAC031 = '1'
AND A.AAC030 >= '2018-01-01 00:00:00'
AND A.AAC001 NOT IN
(SELECT B.AAC001
FROM AC02_TEMP AS B
WHERE B.AAC030 < '2018-01-01 00:00:00');

以下是 explain

CreatingSets (Create sets before main query execution)
  Expression ((Projection + Before ORDER BY))
    Aggregating
      Expression (Before GROUP BY)
        ReadFromMergeTree (default.AC02_TEMP)
        Indexes:
          PrimaryKey
            Keys: 
              AAC001
              AAE200
"            Condition: and((AAC001 notIn 18692488-element set), (AAE200 in ['41', '41']))"
            Parts: 2/2
            Granules: 4821/4821
  CreatingSet (Create set for subquery)
    Expression ((Projection + Before ORDER BY))
      ReadFromMergeTree (default.AA26)
      Indexes:
        PrimaryKey
          Condition: true
          Parts: 1/1
          Granules: 1/1

我是 clickhouse 新手,目前没什么头绪,求大佬帮助 0.0