Let $G$ be a graph with adjacency matrix $A(G)$ and let $D(G)$ be the diagonal matrix of vertex degrees of $G$. For any real $α\in [0,1]$, Nikiforov defined the $A_α$-matrix of a graph $G$ as $A_α(G)=αD(G)+(1-α)A(G)$. The largest eigenvalue of $A_α(G)$ is called the $α$-index or the $A_α$-spectral radius of $G$. A graph is minimally $k$-(edge)-connected if it is $k$-(edge)-connected and deleting any arbitrary chosen edge always leaves a graph which is not $k$-(edge)-connected. In this paper, we characterize the minimally 2-edge-connected graphs and minimally 3-connected graph with given order having the maximum $α$-index for $α\in [\frac{1}{2},1)$, respectively.