Testing for the presence of autocorrelation is a fundamental problem in time series analysis. Classical methods such as the Box-Pierce test rely on the assumption of stationarity, necessitating the removal of non-stationary components such as trends or shifts in the mean prior to application. However, this is not always practical, particularly when the mean structure is complex, such as being piecewise constant with frequent shifts. In this work, we propose a new inferential framework for autocorrelation in time series data under frequent mean shifts. In particular, we introduce a Shift-Immune Portmanteau (SIP) test that reliably tests for autocorrelation and is robust against mean shifts. We illustrate an application of our method to nanopore sequencing data.