Let $G$ be a graph with adjacency matrix $A(G)$ and degree diagonal matrix $D (G)$. In 2017, Nikiforov defined the matrix $A_α(G) = αD(G) + (1-α)A(G)$ for any real $α\in[0,1]$. The largest eigenvalue of $A_α(G)$ is called the $A_α$ spectral radius or the $A_α$-index of $G$. Let $\mathcal{G}_{n,k}^d$ be the set of $k$-connected graphs of order $n$ with diameter $d$. In this paper, we determine the graphs with maximum $A_α$ spectral radius among all graphs in $\mathcal{G}_{n,k}^d$ for any $α\in[0,1)$, where $k\geq2$ and $d\geq2$. We generalizes the results about adjacency matrix of Theorem 3.6 in [P. Huang, W.C. Shiu, P.K. Sun, Linear Algebra Appl., 488 (2016) 350--362] and the results about signless Laplacian matrix of Theorem 3.4 in [P. Huang, J.X. Li, W.C. Shiu, Linear Algebra Appl., 617 (2021) 78--99]. Furthermore, we also obtain the upper and lower bounds of the extremal graph in $\mathcal{G}_{n,k}^d$.