惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

雷峰网
雷峰网
博客园 - 叶小钗
博客园_首页
阮一峰的网络日志
阮一峰的网络日志
D
Docker
J
Java Code Geeks
B
Blog
G
Google Developers Blog
小众软件
小众软件
博客园 - 聂微东
罗磊的独立博客
大猫的无限游戏
大猫的无限游戏
IT之家
IT之家
量子位
WordPress大学
WordPress大学
美团技术团队
钛媒体:引领未来商业与生活新知
钛媒体:引领未来商业与生活新知
宝玉的分享
宝玉的分享
腾讯CDC
Martin Fowler
Martin Fowler
V
Visual Studio Blog
D
DataBreaches.Net
Stack Overflow Blog
Stack Overflow Blog
C
Check Point Blog

cs.DS updates on arXiv.org

PAC Learning with Bandit Feedback: Sharp Sample Complexity in the Realizable Setting Algorithms with Polynomially-Improved Approximation Factors for the $2 \rightarrow q$ Norm, and Applications A computational phase transition for learning-to-sample from Ising models Covering vertices by sequential stars Fermi-Dirac machines as quantizations of neurons A Comprehensive Evaluation of Vertex Elimination Algorithms for Algorithmic Differentiation A Tight Bound on Localization of Electrical Flows Optimal Dimension-Free Sampling for Regularized Classification Reducing the Randomness in Partition Oracles for Bounded Degree Minor-Free Graphs Beyond the Half-Approximation: Fair and Efficient Online Class Matching Efficient Uniform Sampling of Surjections via their Profiles Tractable Maximization of Budgeted Phylogenetic Diversity on Networks Utilizing Node Scanwidth Fairness in Aggregation: Optimal Top-$k$ and Improved Full Ranking Learning-Augmented Online Scheduling with Parsimonious Preemption Entropy Equivalence Testing Lumberjack: Better Differentially Private Random Forests through Heavy Hitter Detection in Trees The Secretary Problem with a Stochastic Precursor Polynomial-Time Robust Multiclass Linear Classification under Gaussian Marginals Efficient Banzhaf-Based Data Valuation for $k$-Nearest Neighbors Classification Block-Sphere Vector Quantization An Approximation Algorithm for Graph Label Selection Iterative Chow Filtering for Learning with Distribution Shift Complexity of Non-Log-Concave Sampling in Fisher Information Stochastic Matching via Local Sparsification Finite Sample Bounds for Learning with Score Matching What is Learnable in Valiant's Theory of the Learnable? Provable Quantization with Randomized Hadamard Transform Min-Max Optimization Requires Exponentially Many Queries Fast and Compact Graph Cuts for the Boykov-Kolmogorov Algorithm A proximal gradient algorithm for composite log-concave sampling
Parallel External Sorting of ASCII Records Using Learned ...
Ani Kristo, Tim Kraska · 2023-05-09 · via cs.DS updates on arXiv.org

External sorting is at the core of many operations in large-scale database systems, such as ordering and aggregation queries for large result sets, building indexes, sort-merge joins, duplicate removal, sharding, and record clustering. Unlike in-memory sorting, these algorithms need to work together with the OS and the filesystem to efficiently utilize system resources and minimize disk I/O. In this paper we describe ELSAR: a parallel external sorting algorithm that uses an innovative paradigm based on a learned data distribution model. The algorithm leverages the model to arrange the input records into mutually exclusive, monotonic, and equi-depth partitions that, once sorted, can simply be concatenated to form the output. This method completely eliminates the need for multi-way file merging, which is typically used in external sorting. We present thorough benchmarks for uniform and skewed datasets in various storage media, where we measure the sorting rates, size scalability, and energy efficiency of ELSAR and other sorting algorithms. We observed that ELSAR has up to 1.65x higher sorting rates than the next-best external sort (Nsort) on SSD drives and 5.31x higher than the GNU coreutils' sort utility on Intel Optane non-volatile memory. In addition, ELSAR supersedes the current winner of the SortBenchmark for the most energy-efficient external string sorting algorithm by an impressive margin of 41%. These results reinforce the premise that novel learning-enhanced algorithms can provide remarkable performance benefits over traditional ones.