惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

Microsoft Azure Blog
Microsoft Azure Blog
aimingoo的专栏
aimingoo的专栏
F
Fortinet All Blogs
Blog — PlanetScale
Blog — PlanetScale
GbyAI
GbyAI
MongoDB | Blog
MongoDB | Blog
月光博客
月光博客
The Cloudflare Blog
量子位
T
Tailwind CSS Blog
Cyber Security Advisories - MS-ISAC
Cyber Security Advisories - MS-ISAC
B
Blog
MyScale Blog
MyScale Blog
T
The Blog of Author Tim Ferriss
The GitHub Blog
The GitHub Blog
G
Google Developers Blog
D
DataBreaches.Net
V
Visual Studio Blog
奇客Solidot–传递最新科技情报
奇客Solidot–传递最新科技情报
Last Week in AI
Last Week in AI
U
Unit 42
博客园 - 聂微东
有赞技术团队
有赞技术团队
A
About on SuperTechFans

Arpit Bhayani

Temporal Primer - Building Long-Running Systems What Matters in Production RAG Structure of Every LLM Chat How LLMs Really Work Your Monolith Is Already A Distributed System Databases Were Not Designed For This BM25 JOIN Algorithms Venting at Work Comes at a Reputation Cost Why Half Your Skills Expire Every Few Years Multi-Paxos - Consensus in Distributed Databases MySQL Replication Internals Bloom Filters When You Increase Kafka Partitions Product Quantization The Q, K, V Matrices The Day I Accidentally Deleted Production How LLM Inference Works What are Blocking Queues and Why We Need Them Heartbeats in Distributed Systems How Writes Work in Apache Cassandra Redis Replication Internals How to Handle Arrogant Colleagues at Work How Does a CDN Handle Content Replication You Can't Fix Everything on Day One When Emotions Spill Over at Work Why gRPC Uses HTTP2 Meetings With No Agenda Are a Waste of Time Career Longevity Beats Constant Job Hopping Stay Relevant at Higher Salary Levels
MongoDB Pagination - Skip is Slow
Arpit Bhayani · 2017-03-23 · via Arpit Bhayani

MongoDB’s cursor object has a method called skip, which as per documentation and definition, controls where MongoDB begins returning results. Thus in combination with function limit, one can easily have paginated results.

I have written a blog post on how you can have Fast and Efficient Pagination in MongoDB.

But while going through the documentation of skip, there is something interesting to notice. There is a small warning in MongoDB documentation, that states

The cursor.skip() method is often expensive because it requires the server to walk from the beginning of the collection or index to get the offset or skip position before beginning to return results. As the offset (e.g. pageNumber above) increases, cursor.skip() will become slower and more CPU intensive. With larger collections, cursor.skip() may become IO bound.

In short, MongoDB has to iterate over documents to skip them. Thus when collection or result set is huge and you need to skip documents for pagination, the call to cursor.skip will be expensive. While going through the source code of skip I found out that it does not use any index and hence gets slower when result set increases in size.

This also implies that if you use skip then the “skipping speed” will not improve even if you index the field.

But what if the size of result set is small? is calling skip still a terrible idea? If skip was so terrible, then MongoDB team and community must had taken that decision long back. But they haven’t … why?

Because it is very efficient and fast for smaller result set. I have taken this opportunity to benchmark and compare the two approach for pagination and there I found out skip and limit based pagination works well for smaller result sets.

In conclusion, skip is not as bad one might think. But you must understand your use case well so as to make an informed decision.