Consider the Monge-Kantorovich problem of transporting densities $ρ_0$ to $ρ_1$ on $\mathbb{R}^d$ with a strictly convex cost function. A popular relaxation of the problem is the one-parameter family called the entropic cost problem. The entropic cost $K_h$, $h>0$, is significantly faster to compute and $h K_h$ is known to converge to the optimal transport cost as $h$ goes to zero. We are interested the rate of convergence. We show that the difference between $K_h$ and $1/h$ times the optimal cost of transport has a pointwise limit when transporting a compactly supported density to another that satisfies a few other technical restrictions. This limit is the relative entropy of $ρ_1$ with respect to a Riemannian volume measure on $\mathbb{R}^d$ that measures the local sensitivity of the transport map. For the quadratic Wasserstein transport, this relative entropy is exactly one half of the difference of entropies of $ρ_1$ and $ρ_0$. In that case we complement the results of Adams et al., Duong et al, and Erbar et al. who all use gamma convergence. More surprisingly, we demonstrate that this difference of two entropies (plus the cost) is also the limit for the Dirichlet transport introduced recently by Pal and Wong. The latter can be thought of as a multiplicative analog of the Wasserstein transport and corresponds to a non-local operator. It hints at an underlying gradient flow of entropy, in the sense of Jordan-Kinderlehrer-Otto, even when the cost function is not a metric. The proofs are based on Gaussian approximations to Schrödinger bridges as $h$ approaches zero.