Bounds on the logarithmic derivatives of the heat kernel on a compact Riemannian manifolds have been long known, and were recently extended, for the log-gradient and log-Hessian, to general complete Riemannian manifolds. Here, we further extend these bounds to incomplete Riemannan manifolds under the least restrictive condition on the distance to infinity available, for derivatives of all orders. Moreover, we consider not only the usual heat kernel associated to the Laplace-Beltrami operator, but we also allow the addition of a conservative vector field. We show that these bounds are sharp in general, even for compact manifolds, and we discuss the difficulties that arise when the operator incorporates non-conservative vector fields or when the Riemannian structure is weakened to a sub-Riemannian structure.