This paper analyzes the correlation matrix between the a priori error and measurement noise vectors for affine projection algorithms (APA). This correlation stems from the dependence between the filter tap estimates and the noise samples, and has a strong influence on the mean square behavior of the algorithm. We show that the correlation matrix is upper triangular, and compute the diagonal elements in closed form, showing that they are independent of the input process statistics. Also, for white inputs we show that the matrix is fully diagonal. These results are valid in the transient and steady states of the algorithm considering a possibly variable step-size. Our only assumption is that the filter order is large compared to the projection order of APA and we make no assumptions on the input signal except for stationarity. Using these results, we perform a steady-state analysis of the algorithm for small step size and provide a new simple closed-form expression for mean-square error, which has comparable or better accuracy to many preexisting expressions, and is much simpler to compute. Finally, we also obtain expressions for the steady-state energy of the other components of the error vector.