We develop a general method for extending Markov processes to a larger state space such that the added points form a polar set. The so obtained extension is an improvement on the standard trivial extension in which case the process is made stuck in the added points, and it renders a new technique of constructing extended solutions to S(P)DEs from all starting points, in such a way that they are solutions at least after any strictly positive time. Concretely, we adopt this strategy to study SDEs with singular coefficients on an infinite dimensional state space (e.g. SPDEs of evolutionary type), for which one often encounters the situation where not every point in the space is allowed as an initial condition. The same can happen when constructing solutions of martingale problems or Markov processes from (generalized) Dirichlet forms, to which our new technique also applies.