We introduce weighted Markovian graphs, a random walk model that decouples the transition dynamics of a Markov chain from (random) edge weights representing the cost of traversing each edge. This decoupling allows us to study the accumulated weight along a path independently of the routing behavior. Crucially, we derive closed-form expressions for the mean and variance of weighted first passage times and weighted Kemeny constants, together with their partial derivatives with respect to both the weight and transition matrices. These results hold for both deterministic and stochastic weights with no distributional assumptions. We demonstrate the framework through two applications, highlighting the dual role of variance. In surveillance networks, we introduce the surprise index, a coefficient-of-variation metric quantifying patrol unpredictability, and show how maximizing it yields policies that are both efficient and hard to anticipate. In traffic networks subject to cascading edge failures, we develop a minimal-intervention framework that adjusts speed limits to preserve connectivity under three increasingly flexible regulatory policies.