惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

aimingoo的专栏
aimingoo的专栏
B
Blog RSS Feed
Recent Announcements
Recent Announcements
Vercel News
Vercel News
M
MIT News - Artificial intelligence
阮一峰的网络日志
阮一峰的网络日志
L
LangChain Blog
Cyber Security Advisories - MS-ISAC
Cyber Security Advisories - MS-ISAC
Microsoft Security Blog
Microsoft Security Blog
H
Help Net Security
T
The Blog of Author Tim Ferriss
Y
Y Combinator Blog
G
Google Developers Blog
罗磊的独立博客
爱范儿
爱范儿
宝玉的分享
宝玉的分享
奇客Solidot–传递最新科技情报
奇客Solidot–传递最新科技情报
酷 壳 – CoolShell
酷 壳 – CoolShell
博客园_首页
S
SegmentFault 最新的问题
WordPress大学
WordPress大学
月光博客
月光博客
人人都是产品经理
人人都是产品经理
Apple Machine Learning Research
Apple Machine Learning Research

stat updates on arXiv.org

Simultaneous Monitoring of Shape and Surface Color via 4D Point Clouds: A Registration-free Approach A Refined Generalization Analysis for Extreme Multi-class Supervised Contrastive Representation Learning Ensemble Distributionally Robust Bayesian Optimisation The Proxy Presumption: From Semantic Embeddings to Valid Social Measures Modulated learning for private and distributed regression with just a single sample per client device Query-efficient model evaluation using cached responses Functional-prior-based approaches to Bayesian PDE-constrained inversion using physics-informed neural networks Optimal Experiments for Partial Causal Effect Identification Order-Agnostic Autoregressive Modelling with Missing Data Grokking or Glitching? How Low-Precision Drives Slingshot Loss Spikes Tuning Derivatives for Causal Fairness in Machine Learning Spherical Flows for Sampling Categorical Data Bayesian Rain Field Reconstruction using Commercial Microwave Links and Diffusion Model Priors GRALIS: A Unified Canonical Framework for Linear Attribution Methods via Riesz Representation Sharp Capacity Thresholds in Linear Associative Memory: From Winner-Take-All to Listwise Retrieval Unified Framework of Distributional Regret in Multi-Armed Bandits and Reinforcement Learning Jacobian-Velocity Bounds for Deployment Risk Under Covariate Drift Self-Attention as Transport: Limits of Symmetric Spectral Diagnostics Perturbation is All You Need for Extrapolating Language Models Adapt or Forget: Provable Tradeoffs Between Adam and SGD in Nonstationary Optimization Realizable Bayes-Consistency for General Metric Losses Graph Convolutional Support Vector Regression for Robust Spatiotemporal Forecasting of Urban Air Pollution Segmenting Human-LLM Co-authored Text via Change Point Detection Stochastic Schrödinger Diffusion Models for Pure-State Ensemble Generation Understanding Self-Supervised Learning via Latent Distribution Matching The Geometric Mechanics of Contrastive Representation Learning: Alignment Potentials, Entropic Dispersion, and Cross-modal Divergence Imbalanced Classification under Capacity Constraints On the Spectral Structure and Objective Equivalence of Orthogonal Multilabel Fisher Discriminants Partially Observed Structural Causal Models First-Order Efficiency for Probabilistic Value Estimation via A Statistical Viewpoint
Gibbs randomness-compression proposition
[Submitted on 29 May 2025 (v1), last revised 16 Jul 2026 (this v · 2025-05-29 · via stat updates on arXiv.org

View PDF HTML (experimental)

Abstract:A proposition that connects randomness and compression is put forward via Gibbs entropy over set of measurement vectors associated with a compression process. In building this connection, we use a performance of a learning task as a probe of compression, over series of compression cycles within a cascade. The Gibbs entropy at each cycle measures the degree of randomness. Consequently a lossy compression process can be seen as an equivalent to {\it directed randomness} that preserves information content under certain bounds of Gibbs entropy and the performance of the learning task. The term directed means we guide the compression process with set of mathematical rules on how to reduce the model size. We formulate this connection with a theorem using a $\delta$ and $\epsilon$ bounds, and demonstrated a logical proof via comonotonic relationship within a very small decrease in compression ratio and the performance. We have showcase the validity of this proposition with a canonical vision task in deep learning with three different model compression processes as {\it a baseline model}. We use the following, simpler to more complex model compression approaches: (1) random pruning, (2) magnitude pruning, and (3) a more complex compression by using dual tomographic compression, which utilizes compressed sensing in dual fashion. We use remaining weights of deep learning network as a measurement vector where we measure the Gibbs entropy. The proposition is supported with the experimental evidence, resulting in very high correlation between learning performance and the Gibbs entropy over compression ratios for all different compression processes. We show case the idea that there is an inherent computable connection between compression probed by performance degradation and randomness from an entropy measure on the learned model.

Submission history

From: Mehmet A. Süzen PhD [view email]
[v1] Thu, 29 May 2025 10:48:35 UTC (367 KB)
[v2] Mon, 30 Jun 2025 12:42:09 UTC (438 KB)
[v3] Mon, 5 Jan 2026 14:07:39 UTC (374 KB)
[v4] Thu, 16 Jul 2026 14:26:46 UTC (478 KB)