In this paper, we develop the theory of flag manifold over a semifield for any Kac-Moody root datum. We show that the flag manifold over a semifield admits a natural action of the monoid over that semifield associated with the Kac-Moody datum and admits a cellular decomposition. This extends the previous work of Lusztig, Postnikov, Rietsch and others on the totally nonnegative flag manifolds (of finite type) and the work of Lusztig, Speyer, Williams on the tropical flag manifolds (of finite type). As a by-product, we prove a conjecture of Lusztig on the duality of totally nonnegative flag manifold of finite type.