Authors:Jinhao You (1), Shuo Lyu (1), Zhuohang Lyu (1), Tanxuan Li (1), Zibo Zhao (1), Jiaxiang Hu (2), Kai Tang (3), Yichen Guo (3) ((1) University of Pennsylvania, (2) University of California, Irvine, (3) Nanyang Technological University)
Abstract:Visual Geometry Grounded Transformer (VGGT) recovers dense 3D scene structure from multi-view images in one forward pass, but quadratic cross-frame attention limits its scalability. Existing training-free accelerators reduce computation uniformly along one axis, missing layer heterogeneity. Our spectral, probing, and causal analyses reveal three regimes: shallow layers lack cross-view structure, middle layers drive cross-view alignment, and deep layers are redundant for dense geometry yet their cross-frame attention remains essential for pose. RegimeVGGT applies layer-wise U-shaped compression along two axes: Saliency-Guided Banded Merging protects geometry- and edge-salient tokens, while Selectively Protected K/V Downsampling preserves cross-frame spatial coverage and the pose-critical path through a phase-shifted spatial grid, a reference-frame anchor, and uncompressed camera/register tokens. Training-free, RegimeVGGT achieves a 6.7x speedup over VGGT* at matched reconstruction quality.
From: Shuo Lyu [view email]
[v1]
Tue, 16 Jun 2026 19:41:23 UTC (11,929 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。