Boolean, free and monotone cumulants as well as relations among them, have proven to be important in the study of non-commutative probability theory. Quite notably, Boolean cumulants were successfully used to study free infinite divisibility via the Boolean Bercovici--Pata bijection. On the other hand, in recent years the concept of infinitesimal non-commutative probability has been developed, together with the notion of infinitesimal cumulants which can be useful in the context of combinatorial questions. In this paper, we show that the known relations among free, Boolean and monotone cumulants still hold in the infinitesimal framework. Our approach is based on the use of Grassmann algebra. Formulas involving infinitesimal cumulants can be obtained by applying a formal derivation to known formulas. The relations between the various types of cumulants turn out to be captured via the shuffle algebra approach to moment-cumulant relations in non-commutative probability theory. In this formulation, (free, Boolean and monotone) cumulants are represented as elements of the Lie algebra of infinitesimal characters over a particular combinatorial Hopf algebra. The latter consists of the graded connected double tensor algebra defined over a non-commutative probability space and is neither commutative nor cocommutative. In this note it is shown how the shuffle algebra approach naturally extends to the notion of infinitesimal non-commutative probability space. The basic step consists in replacing the base field as target space of linear Hopf algebra maps by the Grassmann algebra over the base field. We also consider the infinitesimal analog of the Boolean Bercovici--Pata map.