Holographic MIMO refers to a (possibly large) array with a large number of individually controlled and densely deployed antennas. The objective of this paper is to provide further insight into the use of closely spaced antennas in the uplink and downlink of a multi-user Holographic MIMO system. To this end, we utilize multiport communication theory, which ensures physically consistent uplink and downlink models. We first consider a simple uplink scenario with two side-by-side half-wavelength dipoles, two users, and single-path line-of-sight propagation, and show both analytically and numerically that the array gain and average spectral efficiency strongly depend on the directions from which the signals are received and on the array matching network used. The numerical results are then used to extend the analysis to more practical scenarios involving larger arrays of dipoles (arranged in a uniform linear array) and a larger number of users. The case where the antennas are densely packed in a space-constrained factor form is also considered. It is found that the spectral efficiency increases with decreasing antenna spacing only for arrays of moderate size, e.g. in the order of a few wavelengths. In comparison, larger arrays with closely spaced antennas show only marginal improvements in spectral efficiency compared to half-wavelength arrays.