

























Abstract:Large-scale Nearest Neighbor (NN) search, though widely utilized in the similarity search field, remains challenged by the computational limitations inherent in processing large scale data. In an effort to decrease the computational expense needed, Approximate Nearest Neighbor (ANN) search is often used in applications that do not require the exact similarity search, but instead can rely on an approximation. Product Quantization (PQ) is a memory-efficient ANN effective for clustering all sizes of datasets. Clustering large-scale, high dimensional data requires a heavy computational expense, in both memory-cost and execution time. This work focuses on a unique way to divide and conquer the large scale data in Python using PQ, Inverted Indexing and Dask, combining the results without compromising the accuracy and reducing computational requirements to the level required when using medium-scale data.
| Comments: | To be published in the CSCE 2022 proceedings |
| Subjects: | Machine Learning (cs.LG); Performance (cs.PF) |
| Cite as: | arXiv:2604.21645 [cs.LG] |
| (or arXiv:2604.21645v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2604.21645 arXiv-issued DOI via DataCite (pending registration) |
From: Haley Dozier [view email]
[v1]
Thu, 23 Apr 2026 12:59:30 UTC (790 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理,仅供阅读参考。 原文来自 — 版权归原作者所有。