Given a graph $G$, let $\mathrm{diam}(G)$ be the greatest distance between any two vertices of $G$ which lie in the same connected component, and let $\mathrm{diam}^+(G)$ be the greatest distance between any two vertices of $G$; so $\mathrm{diam}^+(G)=\infty$ if $G$ is not connected. Fix a sequence $(d_1,\ldots,d_n)$ of positive integers, and let $\mathbf{G}$ be a uniformly random connected simple graph with $V(\mathbf{G})=[n]:=\{1,\ldots,n\}$ such that $\mathrm{deg}_{\mathbf{G}}(v)=d_v$ for all $v \in [n]$. We show that, unless a $1-o(1)$ proportion of vertices have degree $2$, then $\mathbb{E}[\mathrm{diam}(\mathbf{G})]=O(\sqrt{n})$. It is not hard to see that this bound is best possible for general degree sequences (and in particular in the case of trees, in which $\sum_{v=1}^n d_v = 2(n-1)$). We also prove that this bound holds without the connectivity constraint. As a key input to the proofs, we show that graphs with minimum degree $3$ are with high probability connected and have logarithmic diameter: if $\min(d_1,\ldots,d_n) \ge 3$ and $\mathbf{G}$ is a uniformly random simple graph with $V(\mathbf{G})=[n]$ such that $\mathrm{deg}_{\mathbf{G}}(v)=d_v$ for all $v \in [n]$, then $\mathrm{diam}^+(\mathbf{G})=$ $O_{\mathbb{P}}(\log n)$; this bound is also best possible.