Lyon's rough paths give an algebraic and analytic framework for Stieltjes integrals in a regime of low regularity where the usual Riemann-Stieltjes integral does not converge. Before we may rigorously define rough paths, we start with the introduction of some basic algebraic terminology. Among them are algebras and coalgebras, two notions which are in some sense dual to each other. As a combination of these notions we obtain bialgebras, and as a special case of them then Hopf algebras, which play a central role in this thesis. After further algebraic preliminaries, we give the examples of Hopf algebras we are interested in. Among them is the example of the polynomial Hopf algebra, whose product is nothing but the usual multiplication of polynomials and whose coproduct can be expressed very simply with the help of a binomial coefficient. We then use the dual pair of tensor Hopf algebras to introduce weakly geometric rough paths, which correspond to notions of Stieltjes integrals satisfying the usual integration by parts rule. For cases like Itô-integration where we need to give up integration by parts, we look at Gubinelli's branched rough paths based on the dual pair of Hopf algebras on trees and forests. Finally, we give some basic concepts of Hairer's theory of regularity structures and use them for a different approach to branched and weakly geometric rough paths. While we first look at a general method described by Hairer to derive a regularity structure from certain Hopf algebras, we then develop a regularity structure based on a formal Picard iteration which is more suitable for dealing with rough differential equations. This work was written as a master's thesis supervised by Peter Friz and Sylvie Paycha and submitted to TU Berlin on July 29 2016. As of 2021, it does not provide novel research material, but can still be used as an algebra-focussed introduction to the subject.