惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

钛媒体:引领未来商业与生活新知
钛媒体:引领未来商业与生活新知
Security Latest
Security Latest
Jina AI
Jina AI
爱范儿
爱范儿
Attack and Defense Labs
Attack and Defense Labs
Recent Commits to openclaw:main
Recent Commits to openclaw:main
GbyAI
GbyAI
N
News | PayPal Newsroom
Last Week in AI
Last Week in AI
V
Visual Studio Blog
SecWiki News
SecWiki News
H
Heimdal Security Blog
D
Darknet – Hacking Tools, Hacker News & Cyber Security
腾讯CDC
L
LINUX DO - 热门话题
I
InfoQ
罗磊的独立博客
博客园_首页
Know Your Adversary
Know Your Adversary
T
The Exploit Database - CXSecurity.com
博客园 - 【当耐特】
阮一峰的网络日志
阮一峰的网络日志
Hacker News - Newest:
Hacker News - Newest: "LLM"
T
Threat Research - Cisco Blogs
美团技术团队
IT之家
IT之家
AWS News Blog
AWS News Blog
OSCHINA 社区最新新闻
OSCHINA 社区最新新闻
N
Netflix TechBlog - Medium
Help Net Security
Help Net Security
S
Secure Thoughts
Microsoft Security Blog
Microsoft Security Blog
Schneier on Security
Schneier on Security
Spread Privacy
Spread Privacy
B
Blog RSS Feed
N
News and Events Feed by Topic
L
Lohrmann on Cybersecurity
Google Online Security Blog
Google Online Security Blog
Microsoft Azure Blog
Microsoft Azure Blog
N
News and Events Feed by Topic
I
Intezer
Blog — PlanetScale
Blog — PlanetScale
云风的 BLOG
云风的 BLOG
A
Arctic Wolf
C
Cyber Attacks, Cyber Crime and Cyber Security
G
Google Developers Blog
The Register - Security
The Register - Security
The GitHub Blog
The GitHub Blog
MyScale Blog
MyScale Blog
F
Full Disclosure

stat.ML updates on arXiv.org

Adaptive multi-fidelity optimization with fast learning rates Enhancing AI and Dynamical Subseasonal Forecasts with Probabilistic Bias Correction Sample Complexity Bounds for Stochastic Shortest Path with a Generative Model The Harder Path: Last Iterate Convergence for Uncoupled Learning in Zero-Sum Games with Bandit Feedback Stylistic-STORM (ST-STORM) : Perceiving the Semantic Nature of Appearance Collective Kernel EFT for Pre-activation ResNets PRIM-cipal components analysis One-Shot Generative Flows: Existence and Obstructions Structural interpretability in SVMs with truncated orthogonal polynomial kernels Amortized Optimal Transport from Sliced Potentials MinShap: A Modified Shapley Value Approach for Feature Selection Unsupervised feature selection using Bayesian Tucker decomposition Multi-User mmWave Beam and Rate Adaptation via Combinatorial Satisficing Bandits Best of both worlds: Stochastic & adversarial best-arm identification Scalable Model-Based Clustering with Sequential Monte Carlo Expert-Guided Class-Conditional Goodness-of-Fit Scores for Interpretable Classification with Informative Missingness: An Application to Seismic Monitoring Lightweight Geometric Adaptation for Training Physics-Informed Neural Networks Gating Enables Curvature: A Geometric Expressivity Gap in Attention Zeroth-Order Optimization at the Edge of Stability Differentially Private Conformal Prediction CLion: Efficient Cautious Lion Optimizer with Enhanced Generalization Generative Augmented Inference Improving Machine Learning Performance with Synthetic Augmentation PAC-MCTS: Bias-Aware Pruning for Robust LLM-Guided Search and Planning Path-Sampled Integrated Gradients Heat and Matérn Kernels on Matchings Doubly Outlier-Robust Online Infinite Hidden Markov Model Momentum Further Constrains Sharpness at the Edge of Stochastic Stability Multistage Conditional Compositional Optimization BOAT: Navigating the Sea of In Silico Predictors for Antibody Design via Multi-Objective Bayesian Optimization Sandpile Economics: Theory, Identification, and Evidence Online learning with noisy side observations Spectral Thompson sampling Covariance-adapting algorithm for semi-bandits with application to sparse rewards Ordinary Least Squares is a Special Case of Transformer Metric-Aware Principal Component Analysis (MAPCA):A Unified Framework for Scale-Invariant Representation Learning Robust Low-Rank Tensor Completion based on M-product with Weighted Correlated Total Variation and Sparse Regularization Joint Representation Learning and Clustering via Gradient-Based Manifold Optimization Universality of Gaussian-Mixture Reverse Kernels in Conditional Diffusion Interpretable and Explainable Surrogate Modeling for Simulations: A State-of-the-Art Survey and Perspectives on Explainable AI for Decision-Making Estimating Continuous Treatment Effects with Two-Stage Kernel Ridge Regression A short proof of near-linear convergence of adaptive gradient descent under fourth-order growth and convexity Some Theoretical Limitations of t-SNE Bias-Corrected Adaptive Conformal Inference for Multi-Horizon Time Series Forecasting Identifiability of Potentially Degenerate Gaussian Mixture Models With Piecewise Affine Mixing Rare Event Analysis via Stochastic Optimal Control Adaptive Learning via Off-Model Training and Importance Sampling for Fully Non-Markovian Optimal Stochastic Control. Complete version Generalization Guarantees on Data-Driven Tuning of Gradient Descent with Langevin Updates Minimizing classical resources in variational measurement-based quantum computation for generative modeling Deep Learning for Sequential Decision Making under Uncertainty: Foundations, Frameworks, and Frontiers ADD for Multi-Bit Image Watermarking Beyond Fixed False Discovery Rates: Post-Hoc Conformal Selection with E-Variables Regional Explanations: Bridging Local and Global Variable Importance ShapShift: Explaining Model Prediction Shifts with Subgroup Conditional Shapley Values Cost-optimal Sequential Testing via Doubly Robust Q-learning Query Lower Bounds for Diffusion Sampling Tail-Aware Information-Theoretic Generalization for RLHF and SGLD Beyond Augmented-Action Surrogates for Multi-Expert Learning-to-Defer Hierarchical Kernel Transformer: Multi-Scale Attention with an Information-Theoretic Approximation Analysis Policy-Aware Design of Large-Scale Factorial Experiments Towards Verified and Targeted Explanations through Formal Methods Portfolio Optimization Proxies under Label Scarcity and Regime Shifts via Bayesian and Deterministic Students under Semi-Supervised Sandwich Training Spectral methods: crucial for machine learning, natural for quantum computers? The Devil Is in Gradient Entanglement: Energy-Aware Gradient Coordinator for Robust Generalized Category Discovery A Tutorial Review of Bayesian Optimization with Gaussian Processes to Accelerate Stationary Point Searches Certified and accurate computation of function space norms of deep neural networks Mini-Batch Covariance, Diffusion Limits, and Oracle Complexity in Stochastic Gradient Descent: A Sampling-Design Perspective Conformal Policy Control Diagnostics for Individual-Level Prediction Instability in Machine Learning for Healthcare Neural Networks With Dense Weights Are Not Universal Approximators Continuous-time reinforcement learning: ellipticity enables model-free value function approximation Scalable spatial point process models for forensic footwear analysis A Review of Diffusion-based Simulation-Based Inference: Foundations and Applications in Non-Ideal Data Scenarios Active Learning with Selective Time-Step Acquisition for PDEs Joint Score-Threshold Optimization for Interpretable Risk Assessment Revisiting Entropy Regularization: Adaptive Coefficient Unlocks Its Potential for LLM Reinforcement Learning Discrete Guidance Matching: Exact Guidance for Discrete Flow Matching PnP-CM: Consistency Models as Plug-and-Play Priors for Inverse Problems Online Distributionally Robust LLM Alignment via Regression to Relative Reward Heavy-Tailed Class-Conditional Priors for Long-Tailed Generative Modeling Random Walk Learning and the Pac-Man Attack Sequential Regression Learning with Randomized Algorithms Diagnosing and Improving Diffusion Models by Estimating the Optimal Loss Value Random Matrix Theory for Deep Learning: Beyond Eigenvalues of Linear Models Scalable Spatiotemporal Inference with Biased Scan Attention Transformer Neural Processes Towards AI-assisted Neutrino Flavor Theory Design Towards Reasonable Concept Bottleneck Models Practical estimation of the optimal classification error with soft labels and calibration Flow-based Generative Modeling of Potential Outcomes and Counterfactuals The Gaussian Latent Machine: Efficient Prior and Posterior Sampling for Inverse Problems Two-Dimensional Deep ReLU CNN Approximation for Korobov Functions: A Constructive Approach FSPO: Few-Shot Optimization of Synthetic Preferences Personalizes to Real Users Identifying Information from Observations with Uncertainty and Novelty A ghost mechanism: An analytical model of abrupt learning in recurrent networks A Multiparty Homomorphic Encryption Approach to Confidential Federated Kaplan Meier Survival Analysis Large Language Models for Market Research: A Data-augmentation Approach Transformer Neural Processes - Kernel Regression FIT-GNN: Faster Inference Time for GNNs that 'FIT' in Memory Using Coarsening Estimating Joint Interventional Distributions from Marginal Interventional Data Nonparametric Sparse Online Learning of the Koopman Operator
OPORP: One Permutation + One Random Projection
Ping Li, Xiaoyun Li · 2023-02-07 · via stat.ML updates on arXiv.org

Consider two $D$-dimensional data vectors (e.g., embeddings): $u, v$. In many embedding-based retrieval (EBR) applications where the vectors are generated from trained models, $D=256\sim 1024$ are common. In this paper, OPORP (one permutation + one random projection) uses a variant of the ``count-sketch'' type of data structures for achieving data reduction/compression. With OPORP, we first apply a permutation on the data vectors. A random vector $r$ is generated i.i.d. with moments: $E(r_i) = 0, E(r_i^2)=1, E(r_i^3) =0, E(r_i^4)=s$. We multiply (as dot product) $r$ with all permuted data vectors. Then we break the $D$ columns into $k$ equal-length bins and aggregate (i.e., sum) the values in each bin to obtain $k$ samples from each data vector. One crucial step is to normalize the $k$ samples to the unit $l_2$ norm. We show that the estimation variance is essentially: $(s-1)A + \frac{D-k}{D-1}\frac{1}{k}\left[ (1-ρ^2)^2 -2A\right]$, where $A\geq 0$ is a function of the data ($u,v$). This formula reveals several key properties: (1) We need $s=1$. (2) The factor $\frac{D-k}{D-1}$ can be highly beneficial in reducing variances. (3) The term $\frac{1}{k}(1-ρ^2)^2$ is a substantial improvement compared with $\frac{1}{k}(1+ρ^2)$, which corresponds to the un-normalized estimator. We illustrate that by letting the $k$ in OPORP to be $k=1$ and repeat the procedure $m$ times, we exactly recover the work of ``very spars random projections'' (VSRP). This immediately leads to a normalized estimator for VSRP which substantially improves the original estimator of VSRP. In summary, with OPORP, the two key steps: (i) the normalization and (ii) the fixed-length binning scheme, have considerably improved the accuracy in estimating the cosine similarity, which is a routine (and crucial) task in modern embedding-based retrieval (EBR) applications.