The main challenges in designing downlink coordinated multicast beamforming in massive multiple-input multiple output (MIMO) cellular networks are the complex computational solutions and significant fronthaul overhead for centralized coordination. This paper proposes a coordinated multicast beamforming solution that is both computation and communication efficient. For joint BS coordination with individual base station transmit power budgets, we first obtain the optimal structure of coordinated multicast beamforming. It reveals that the beamformer at each BS is naturally distributed and only depends on the local channel state information (CSI) at its serving BS. Moreover, the optimal beamformer is a weighted minimum mean square error (MMSE) beamformer with a low-dimensional structure of unknown weights to be optimized, independent of the number of BS antennas. Utilizing the optimal structural properties, we propose fast algorithms to determine the unknown parameters for the optimal beamformer. The main iterative algorithm decomposes the problem into small subproblems, yielding only closed/semi-closed form updates. Furthermore, we propose a semi-distributed computing approach for the proposed algorithm that allows each BS to compute its beamformer based on the local CSI without the need for global CSI sharing, resulting in the fronthaul overhead independent of the number of BS antennas. We further extend our results to the design under the imperfect CSI and other coordination scenarios. Simulation results demonstrate that our proposed methods can achieve near-optimal performance with significantly lower computational time for massive MIMO systems than the conventional approaches.