惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

freeCodeCamp Programming Tutorials: Python, JavaScript, Git & More
S
SegmentFault 最新的问题
N
Netflix TechBlog - Medium
Vercel News
Vercel News
F
Fortinet All Blogs
量子位
博客园 - Franky
酷 壳 – CoolShell
酷 壳 – CoolShell
Cyber Security Advisories - MS-ISAC
Cyber Security Advisories - MS-ISAC
MongoDB | Blog
MongoDB | Blog
Y
Y Combinator Blog
GbyAI
GbyAI
博客园 - 三生石上(FineUI控件)
Apple Machine Learning Research
Apple Machine Learning Research
爱范儿
爱范儿
月光博客
月光博客
Recent Announcements
Recent Announcements
人人都是产品经理
人人都是产品经理
Hugging Face - Blog
Hugging Face - Blog
D
DataBreaches.Net
H
Help Net Security
阮一峰的网络日志
阮一峰的网络日志
D
Docker
WordPress大学
WordPress大学

cs.DC updates on arXiv.org

DUAL-BLADE: Dual-Path NVMe-Direct KV-Cache Offloading for Edge LLM Inference Progressive Semantic Communication for Efficient Edge-Cloud Vision-Language Models Efficient, VRAM-Constrained xLM Inference on Clients Folding Tensor and Sequence Parallelism for Memory-Efficient Transformer Training & Inference DORA: A Scalable Asynchronous Reinforcement Learning System for Language Model Training AMMA: A Multi-Chiplet Memory-Centric Architecture for Low-Latency 1M Context Attention Serving RaMP: Runtime-Aware Megakernel Polymorphism for Mixture-of-Experts Spark Policy Toolkit: Semantic Contracts and Scalable Execution for Policy Learning in Spark Internet of Everything in the 6G Era: Paradigms, Enablers, Potentials and Future Directions PolyKV: A Shared Asymmetrically-Compressed KV Cache Pool for Multi-Agent LLM Inference A Survey on Split Learning for LLM Fine-Tuning: Models, Systems, and Privacy Optimizations ITAS: A Multi-Agent Architecture for LLM-Based Intelligent Tutoring Latency and Cost of Multi-Agent Intelligent Tutoring at Scale TACO: Efficient Communication Compression of Intermediate Tensors for Scalable Tensor-Parallel LLM Training FreeScale: Distributed Training for Sequence Recommendation Models with Minimal Scaling Cost CommFuse: Hiding Tail Latency via Communication Decomposition and Fusion for Distributed LLM Training A Taxonomy and Resolution Strategy for Client-Level Disagreements in Federated Learning Usable Agent Discovery for Decentralized AI Systems Cloud to Edge: Benchmarking LLM Inference On Hardware-Accelerated Single-Board Computers Data-Free Contribution Estimation in Federated Learning using Gradient von Neumann Entropy Shard the Gradient, Scale the Model: Serverless Federated Aggregation via Gradient Partitioning Promoting Simple Agents: Ensemble Methods for Event-Log Prediction GraphLeap: Decoupling Graph Construction and Convolution for Vision GNN Acceleration on FPGA AGNT2: Autonomous Agent Economies on Interaction-Optimized Layer 2 Infrastructure FedSIR: Spectral Client Identification and Relabeling for Federated Learning with Noisy Labels Stream-CQSA: Avoiding Out-of-Memory in Attention Computation via Flexible Workload Scheduling A Delta-Aware Orchestration Framework for Scalable Multi-Agent Edge Computing Federated Learning over Blockchain-Enabled Cloud Infrastructure Optimal Routing for Federated Learning over Dynamic Satellite Networks: Tractable or Not? Sherpa.ai Privacy-Preserving Multi-Party Entity Alignment without Intersection Disclosure for Noisy Identifiers
Bandwidth Cost of Code Conversions in Distributed Storage...
Francisco Maturana, K. V. Rashmi · 2020-08-28 · via cs.DC updates on arXiv.org

Erasure codes have become an integral part of distributed storage systems as a tool for providing data reliability and durability under the constant threat of device failures. In such systems, an $[n, k]$ code over a finite field $\mathbb{F}_q$ encodes $k$ message symbols into $n$ codeword symbols from $\mathbb{F}_q$ which are then stored on $n$ different nodes in the system. Recent work has shown that significant savings in storage space can be obtained by tuning $n$ and $k$ to variations in device failure rates. Such a tuning necessitates code conversion: the process of converting already encoded data under an initial $[n^I, k^I]$ code to its equivalent under a final $[n^F, k^F]$ code. The default approach to conversion is to reencode data, which places significant burden on system resources. Convertible codes are a recently proposed class of codes for enabling resource-efficient conversions. Existing work on convertible codes has focused on minimizing access cost, i.e., the number of code symbols accessed during conversion. Bandwidth, which corresponds to the amount of data read and transferred, is another important resource to optimize. In this paper, we initiate the study on the fundamental limits on bandwidth used during code conversion and present constructions for bandwidth-optimal convertible codes. First, we model the code conversion problem using network information flow graphs with variable capacity edges. Second, focusing on MDS codes and an important parameter regime called the merge regime, we derive tight lower bounds on the bandwidth cost of conversion. The derived bounds show that bandwidth cost can be significantly reduced even in regimes where access cost cannot be reduced as compared to the default approach. Third, we present a new construction for MDS convertible codes which matches the proposed lower bound and is thus bandwidth-optimal during conversion.