For a code $C$ in a space with maximal distance $n$, we say that $C$ has symmetric distances if its distance set $S(C)$ is symmetric with respect to $n / 2$. In this paper, we prove that if $C$ is a binary code with length $2n$, constant weight $n$ and symmetric distances, then \[ |C| \leq \binom{2 n - 1}{|S(C)|}. \] This result can be interpreted using the language of Johnson association schemes. More generally, we give a framework to study codes with symmetric distances in Q-bipartite Q-polynomial association schemes, and provide upper bounds for such codes. Moreover, we use number theoretic techniques to determine when the equality holds.