In this paper, we introduce \emph{$\ell^p$-information geometry}, an infinite-dimensional framework that shares key features with the geometry of the space of probability densities \( \mathrm{Dens}(M) \) on a closed manifold, while also incorporating aspects of measure-valued information geometry. We define the \emph{$\ell^2$-probability simplex} with a noncanonical differentiable structure induced via the \emph{$q$-root transform} from an open subset of the \( \ell^q \)-sphere. This choice makes the \(q\)-root transform an \emph{isometry} and allows us to construct the \(\ell^2\)- and \(\ell^q\)-Fisher--Rao geometries, including \emph{Amari--Čencov \(α\)-connections} and a \emph{Chern connection} in the \(\ell^q\)-setting. We then apply this framework to an infinite-dimensional linear optimization problem. We show that the corresponding gradient flow with respect to the \(\ell^2\)--Fisher--Rao metric can be solved explicitly, converges to a maximizer under a natural monotonicity assumption, and admits an interpretation as the geodesic flow of an \emph{exponential connection}. In particular, we prove that this \(e\)-connection is \emph{geodesically complete}. We further relate these flows to a \emph{completely integrable Hamiltonian system} through a \emph{momentum map} associated with a Hamiltonian torus action on infinite-dimensional complex projective space. Finally, inspired by the \(\ell^2\)-theory, we outline an analogous Fisher--Rao geometry for \( \mathrm{Dens}(M) \) on possibly noncompact Riemannian manifolds, showing that, with a suitable spherical differentiable structure, the square-root transform remains an \emph{isometry}.