On a compact Riemannian manifold $M,$ we show that the Riemannian distance function $d(x,y)$ can be explicitly reconstructed from suitable asymptotics of the expected signature of Brownian bridge from $x$ to $y$. In addition, by looking into the asymptotic expansion of the fourth level expected signature of the Brownian loop based at $x\in M$, one can explicitly reconstruct both intrinsic (Ricci curvature) and extrinsic (second fundamental form) curvature properties of $M$ at $x$. As independent interest, we also derive the intrinsic PDE for the expected Brownian signature dynamics on $M$ from the perspective of the Eells-Elworthy-Malliavin horizontal lifting.