Analog computing has been recently revived due to its potential for energy-efficient and highly parallel computations. In this two-part paper, we explore analog computers that linearly process microwave signals, named microwave linear analog computers (MiLACs), and their applications in signal processing and communications. In Part I of this paper, we model a MiLAC as a multiport microwave network with tunable impedance components, enabling the execution of mathematical operations by reconfiguring the microwave network and applying input signals at its ports. We demonstrate that a MiLAC can efficiently compute the linear minimum mean square error (LMMSE) estimator and matrix inversion, with remarkably low computational complexity. Specifically, a matrix can be inverted with complexity growing with the square of its size. We also show how a MiLAC can be used jointly with digital operations to implement sophisticated algorithms such as the Kalman filter. To enhance practicability, we propose a design of MiLAC based on lossless impedance components, reducing power consumption and eliminating the need for costly active components. In Part II of this paper, we investigate the applications of MiLACs in wireless communications, highlighting their potential to enable future wireless systems by executing computations and beamforming in the analog domain.