惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

量子位
I
InfoQ
人人都是产品经理
人人都是产品经理
博客园 - 三生石上(FineUI控件)
爱范儿
爱范儿
Hugging Face - Blog
Hugging Face - Blog
freeCodeCamp Programming Tutorials: Python, JavaScript, Git & More
S
SegmentFault 最新的问题
美团技术团队
小众软件
小众软件
Blog — PlanetScale
Blog — PlanetScale
Jina AI
Jina AI
aimingoo的专栏
aimingoo的专栏
H
Help Net Security
Last Week in AI
Last Week in AI
博客园_首页
奇客Solidot–传递最新科技情报
奇客Solidot–传递最新科技情报
L
LangChain Blog
云风的 BLOG
云风的 BLOG
Martin Fowler
Martin Fowler
宝玉的分享
宝玉的分享
G
Google Developers Blog
博客园 - 叶小钗
博客园 - Franky

cs.IT updates on arXiv.org

Theoretical Limits of Language Model Alignment $f$-Divergence Regularized RLHF: Two Tales of Sampling and Unified Analyses A Unified Measure-Theoretic View of Diffusion, Score-Based, and Flow Matching Generative Models When Can Voting Help, Hurt, or Change Course? Exact Structure of Binary Test-Time Aggregation When Semantic Communication Meets Queueing: Cross-Layer Latency and Task Fidelity Optimization Convexity in Disguise: A Theoretical Framework for Nonconvex Low-Rank Matrix Estimation Conditional Diffusion Under Linear Constraints: Langevin Mixing and Information-Theoretic Guarantees Sharp Capacity Thresholds in Linear Associative Memory: From Winner-Take-All to Listwise Retrieval Expert Routing for Communication-Efficient MoE via Finite Expert Banks Contextual Memory-Enhanced Source Coding for Low-SNR Communications Realizable Bayes-Consistency for General Metric Losses Leveraging Code Automorphisms for Improved Syndrome-Based Neural Decoding A Hierarchical Sampling Framework for bounding the Generalization Error of Federated Learning Dueling DDQN-Based Adaptive Multi-Objective Handover Optimization for LEO Satellite Networks The Causal Description Gap: Information-Theoretic Separations Across Pearl's Hierarchy Optimization of CV-QKD Under Practical Constraints Benchmarking Wireless Representations: High-Dimensional vs. Compressed Embeddings for Efficiency and Robustness Real-Time Text Transmission via LLM-Based Entropy Coding over Fixed-Rate Channels SwiftChannel: Algorithm-Hardware Co-Design for Deep Learning-Based 5G Channel Estimation Evolving Token Communication with Parametric Memory Network Remote Action Generation: Remote Control with Minimal Communication The (Marginal) Value of a Search Ad: An Online Causal Framework for Repeated Second-price Auctions Stabilizing Private LASSO under Heterogeneous Covariates via Anisotropic Objective Perturbation Linear-Readout Floors and Threshold Recovery in Computation in Superposition Soft Graph Diffusion Transformer for MIMO Detection Hierarchical Federated Learning for Networked AI: From Communication Saving to Architecture-Aware Design Exponential families from a single KL identity MIFair: A Mutual-Information Framework for Intersectionality and Multiclass Fairness Diffusion-OAMP for Joint Image Compression and Wireless Transmission Decoupled Descent: Exact Test Error Tracking Via Approximate Message Passing
Quantify the Causes of Causal Emergence: Critical Conditi...
Liye Jia, Fengyufan Yang, Ka Lok Man, Erick Purwanto, Sheng-Uei · 2022-12-03 · via cs.IT updates on arXiv.org

Beneficial to advanced computing devices, models with massive parameters are increasingly employed to extract more information to enhance the precision in describing and predicting the patterns of objective systems. This phenomenon is particularly pronounced in research domains associated with deep learning. However, investigations of causal relationships based on statistical and informational theories have posed an interesting and valuable challenge to large-scale models in the recent decade. Macroscopic models with fewer parameters can outperform their microscopic counterparts with more parameters in effectively representing the system. This valuable situation is called "Causal Emergence." This paper introduces a quantification framework, according to the Effective Information and Transition Probability Matrix, for assessing numerical conditions of Causal Emergence as theoretical constraints of its occurrence. Specifically, our results quantitatively prove the cause of Causal Emergence. By a particular coarse-graining strategy, optimizing uncertainty and asymmetry within the model's causal structure is significantly more influential than losing maximum information due to variations in model scales. Moreover, by delving into the potential exhibited by Partial Information Decomposition and Deep Learning networks in the study of Causal Emergence, we discuss potential application scenarios where our quantification framework could play a role in future investigations of Causal Emergence.