惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

让小产品的独立变现更简单 - ezindie.com
让小产品的独立变现更简单 - ezindie.com
U
Unit 42
IT之家
IT之家
Y
Y Combinator Blog
T
Tailwind CSS Blog
B
Blog
大猫的无限游戏
大猫的无限游戏
博客园 - 叶小钗
Jina AI
Jina AI
freeCodeCamp Programming Tutorials: Python, JavaScript, Git & More
I
InfoQ
J
Java Code Geeks
F
Fortinet All Blogs
T
The Blog of Author Tim Ferriss
OSCHINA 社区最新新闻
OSCHINA 社区最新新闻
H
Hackread – Cybersecurity News, Data Breaches, AI and More
人人都是产品经理
人人都是产品经理
腾讯CDC
Hugging Face - Blog
Hugging Face - Blog
GbyAI
GbyAI
博客园 - 司徒正美
The GitHub Blog
The GitHub Blog
酷 壳 – CoolShell
酷 壳 – CoolShell
L
LangChain Blog

博客园 - papering

由于系统缓冲区空间不足或队列已满,不能执行套接字上的操作 string byte 内存 硬盘 CAS ABA AtomicLong mysql 索引长度限制 docker network FTP Connection Modes (Active vs. Passive) 字符长度 mysql 【车载架构】AUTOSAR CP系列之27:系统内核之 WdgM 看门狗管理 单目录文件数限制 在联表 join 时 使用 case when 按不同的条件关联不同的行 JDK21 Process Reaper TLS 栈溢出循环故障深度分析 栈溢出 写入相邻的内存区域 GCC 的 -fstack-protector 会在每个函数的栈帧底部放一个随机“金丝雀值”,函数返回前检查它是否被改掉,以此检测栈缓冲区溢出。 潜伏5年!GitLab高危RCE漏洞爆发,低权限即可控服 Redis 漏洞分析——lua 脚本篇 Redis guarantees the script's atomic execution. AMQP 0-9-1 连接通过 信道 进行多路复用,信道可以被认为是“共享单一 TCP 连接的轻量级连接”。 WSAETIMEDOUT WSAEACCES This module supports asynchronous I/O on multiple file descriptors. 设置其优先级值的线程的句柄 QT 主线程 优化 卡顿 主线程上的同步重活 防重入:进行中直接 return,避免连点双开。 把「一次性任务」收到 ThreadPoolExecutor(max_workers=2~3),限制峰值线程数 统一 Activity 浮层 去掉连环成功弹窗 为 with语句上下文提供的工具 懒加载 IDE发现 import Canvas 指纹 魔改chromium源码——CDP(Chrome DevTools Protocol)检测01 whether the browser environment is controlled by a robot. chromium指纹魔改 对拷线 rpa 任务编排 a JSON formatted stream to ``fp`` “幽灵字符”问题 浏览器背后的黑科技 多进程 多线程
search_after 深度分页 Scrolling is not intended for real ...
papering · 2026-04-20 · via 博客园 - papering

https://www.elastic.co/docs/reference/elasticsearch/rest-apis/paginate-search-results

By default, searches return the top 10 matching hits. To page through a larger set of results, you can use the search API's from and size parameters. The from parameter defines the number of hits to skip, defaulting to 0. The size parameter is the maximum number of hits to return. Together, these two parameters define a page of results.

GET /_search
{
  "from": 5,
  "size": 20,
  "query": {
    "match": {
      "user.id": "kimchy"
    }
  }
}

Avoid using from and size to page too deeply or request too many results at once. Search requests usually span multiple shards. Each shard must load its requested hits and the hits for any previous pages into memory. For deep pages or large sets of results, these operations can significantly increase memory and CPU usage, resulting in degraded performance or node failures.

By default, you cannot use from and size to page through more than 10,000 hits. This limit is a safeguard set by the index.max_result_window index setting. If you need to page through more than 10,000 hits, use the search_after parameter instead.

request

GET twitter/_search
{
    "query": {
        "match": {
            "title": "elasticsearch"
        }
    },
    "sort": [
        {"date": "asc"},
        {"tie_breaker_id": "asc"}
    ]
}

response

{
  "took" : 17,
  "timed_out" : false,
  "_shards" : ...,
  "hits" : {
    "total" : ...,
    "max_score" : null,
    "hits" : [
      ...
      {
        "_index" : "twitter",
        "_id" : "654322",
        "_score" : null,
        "_source" : ...,
        "sort" : [
          1463538855,
          "654322"
        ]
      },
      {
        "_index" : "twitter",
        "_id" : "654323",
        "_score" : null,
        "_source" : ...,
        "sort" : [
          1463538857,
          "654323"
        ]
      }
    ]
  }
}

search after

GET twitter/_search
{
    "query": {
        "match": {
            "title": "elasticsearch"
        }
    },
    "search_after": [1463538857, "654323"],
    "sort": [
        {"date": "asc"},
        {"tie_breaker_id": "asc"}
    ]
}

Repeat this process by updating the search_after array every time you retrieve a new page of results. If a refresh occurs between these requests, the order of your results may change, causing inconsistent results across pages. To prevent this, you can create a point in time (PIT) to preserve the current index state over your searches.

request

POST /my-index-000001/_pit?keep_alive=1m
{
  "id": "46ToAwMDaWR5BXV1aWQyKwZub2RlXzMAAAAAAAAAACoBYwADaWR4BXV1aWQxAgZub2RlXzEAAAAAAAAAAAEBYQADaWR5BXV1aWQyKgZub2RlXzIAAAAAAAAAAAwBYgACBXV1aWQyAAAFdXVpZDEAAQltYXRjaF9hbGw_gAAAAA==",
  "_shards": ...
}

request

GET /_search
{
  "size": 10000,
  "query": {
    "match" : {
      "user.id" : "elkbee"
    }
  },
  "pit": {
    "id":  "46ToAwMDaWR5BXV1aWQyKwZub2RlXzMAAAAAAAAAACoBYwADaWR4BXV1aWQxAgZub2RlXzEAAAAAAAAAAAEBYQADaWR5BXV1aWQyKgZub2RlXzIAAAAAAAAAAAwBYgACBXV1aWQyAAAFdXVpZDEAAQltYXRjaF9hbGw_gAAAAA==",
    "keep_alive": "1m"
  },
  "sort": [
    {"@timestamp": {"order": "asc", "format": "strict_date_optional_time_nanos", "numeric_type" : "date_nanos" }}
  ]
}

We no longer recommend using the scroll API for deep pagination. If you need to preserve the index state while paging through more than 10,000 hits, use the search_after parameter with a point in time (PIT).

While a search request returns a single page of results, the scroll API can be used to retrieve large numbers of results (or even all results) from a single search request, in much the same way as you would use a cursor on a traditional database.

Scrolling is not intended for real time user requests, but rather for processing large amounts of data, e.g. in order to reindex the contents of one data stream or index into a new data stream or index with a different configuration.