Consider an undirected weighted graph G=(V,E) with |V|=n and |E|=m, where each vertex v is assigned a label from a set L of \ell labels. We show how to construct a compact distance oracle that can answer queries of the form: "what is the distance from v to the closest lambda-labeled node" for a given node v in V and label lambda in L. This problem was introduced by Hermelin, Levy, Weimann and Yuster [ICALP 2011] where they present several results for this problem. In the first result, they show how to construct a vertex-label distance oracle of expected size O(kn^{1+1/k}) with stretch (4k - 5) and query time O(k). In a second result, they show how to reduce the size of the data structure to O(kn \ell^{1/k}) at the expense of a huge stretch, the stretch of this construction grows exponentially in k, (2^k-1). In the third result they present a dynamic vertex-label distance oracle that is capable of handling label changes in a sub-linear time. The stretch of this construction is also exponential in k, (2 3^{k-1}+1). We manage to significantly improve the stretch of their constructions, reducing the dependence on k from exponential to polynomial (4k-5), without requiring any tradeoff regarding any of the other variables. In addition, we introduce the notion of vertex-label spanners: subgraphs that preserve distances between every node v and label lambda. We present an efficient construction for vertex-label spanners with stretch-size tradeoff close to optimal.