Mathematical definitions of polyhedrons and perceptron networks are discussed. The formalization of polyhedrons is done in a rather traditional way. For networks, previously proposed systems are developed. Perceptron networks in disjunctive normal form (DNF) and conjunctive normal forms (CNF) are introduced. The main theme is that single output perceptron neural networks and characteristic functions of polyhedrons are one and the same class of functions. A rigorous formulation and proof that three layers suffice is obtained. The various constructions and results are among several steps required for algorithms that replace incremental and statistical learning with more efficient, direct and exact geometric methods for calculation of perceptron architecture and weights.