In this article, nonstationary mixing and source models are combined for developing new fast and accurate algorithms for Independent Component or Vector Extraction (ICE/IVE), one of which stands for a new extension of the well-known FastICA. This model allows for a moving source-of-interest (SOI) whose distribution on short intervals can be (non-)circular (non-)Gaussian. A particular Gaussian source model assuming tridiagonal covariance matrix structures is proposed. It is shown to be beneficial in the frequency-domain speaker extraction problem. The algorithms are verified in simulations. In comparison to the state-of-the-art algorithms, they show superior performance in terms of convergence speed and extraction accuracy.