In studies of time-to-event outcomes with unmeasured heterogeneity, the hazard ratio for treatment is known to have a complex causal interpretation. Accelerated failure time (AFT) models, which assess the effect on the survival time ratio scale, are often suggested as a better alternative because they model a parameter with direct causal interpretation while allowing straightforward adjustment for measured confounders. In this work, we formalize the causal interpretation of the acceleration factor in AFT models using structural causal models and data under independent censoring. We prove that the acceleration factor is a valid causal effect measure, even in the presence of frailty and treatment effect heterogeneity. Through simulations, we show that the acceleration factor better captures the causal effect than the hazard ratio when both AFT and conditional proportional hazards models apply. Additionally, we extend the interpretation to systems with time-dependent acceleration factors, illustrating the impossibility of distinguishing between a time-varying homogeneous effect and unmeasured effect heterogeneity. While the causal interpretation of acceleration factors is promising, we caution practitioners about potential challenges for the interpretation in the presence of effect heterogeneity.