In this paper we introduce and study Brownian motion on state spaces with varying dimension. Starting with a concrete case of such state spaces that models a big square with a flag pole, we construct a Brownian motion on it and study how heat propagates on such a space. We derive sharp two-sided global estimates on its transition density functions (also called heat kernel). These two-sided estimates are of Gaussian type, but the measure on the underlying state space does not satisfy volume doubling property. Parabolic Harnack inequality fails for such a process. Nevertheless, we show Holder regularity holds for its parabolic functions. We also derive the Green function estimates for this process on bounded smooth domains. Brownian motion on some other state spaces with varying dimension are also constructed and studied in this paper.