This article develops, via the perspective of (arithmetic) algebraic geometry and category theory, different aspects of geometry of information. First, we describe in the terms of Eilenberg--Moore algebras over a Giry monad, the collection $Cap_n$ of all probability distributions on the measurable space $(Ω_n, \mathcal{A})$ (where $Ω$ is discrete with $n$ issues) and it turns out that there exists an embedding relation of Segre type among the product of $Cap_n$'s. We unravel hidden symmetries of these type of embeddings and show that there exists a hypercubic relation. Secondly, we show that the Manin conjecture -- initially defined concerning the diophantine geometry of Fano varieties -- is true in the case of exponential statistical manifolds, defined over a discrete sample space. Thirdly, we introduce a modified version of the parenthesised braids ($\mathbf{mPaB}$), which forms a key tool in code-correction. This modified version $\mathbf{mPaB}$ presents all types of mistakes that could occur during a transmission process. We show that the standard parenthesised braids $\mathbf{PaB}$ form a full subcategory of $\mathbf{mPaB}$. We discuss the role of the Grothendieck--Teichmüller group in relation to the modified parenthesised braids. Finally, we prove that the motivic Galois group is contained in the automorphism $Aut(\widehat{\mathbf{mPaB}}).$ We conclude by presenting an open question concerning rational points, Commutative Moufang Loops and information geometry.