Constraint Handling Rules is an effective concurrent declarative programming language and a versatile computational logic formalism. CHR programs consist of guarded reactive rules that transform multisets of constraints. One of the main features of CHR is its inherent concurrency. Intuitively, rules can be applied to parts of a multiset in parallel. In this comprehensive survey, we give an overview of concurrent and parallel as well as distributed CHR semantics, standard and more exotic, that have been proposed over the years at various levels of refinement. These semantics range from the abstract to the concrete. They are related by formal soundness results. Their correctness is established as correspondence between parallel and sequential computations. We present common concise sample CHR programs that have been widely used in experiments and benchmarks. We review parallel CHR implementations in software and hardware. The experimental results obtained show a consistent parallel speedup. Most implementations are available online. The CHR formalism can also be used to implement and reason with models for concurrency. To this end, the Software Transaction Model, the Actor Model, Colored Petri Nets and the Join-Calculus have been faithfully encoded in CHR. Under consideration in Theory and Practice of Logic Programming (TPLP).