The least mean squares (LMS) filter is often derived via the Wiener filter solution. For a system identification scenario, such a derivation makes it hard to incorporate prior information on the system's impulse response. We present an alternative way based on the maximum a posteriori solution, which allows developing a Knowledge-Aided Kaczmarz algorithm. Based on this Knowledge-Aided Kaczmarz we formulate a Knowledge-Aided LMS filter. Both algorithms allow incorporating the prior mean and covariance matrix on the parameter to be estimated. The algorithms use this prior information in addition to the measurement information in the gradient for the iterative update of their estimates. We analyze the convergence of the algorithms and show simulation results on their performance. As expected, reliable prior information allows improving the performance of the algorithms for low signal-to-noise (SNR) scenarios. The results show that the presented algorithms can nearly achieve the optimal maximum a posteriori (MAP) performance.