The impacts of channel estimation errors, inter-cell interference, phase adjustment cost, and computation cost on an intelligent reflecting surface (IRS)-assisted system are severe in practice but have been ignored for simplicity in most existing works. In this paper, we investigate a multi-antenna base station (BS) serving a single-antenna user with the help of a multi-element IRS in the presence of channel estimation errors and inter-cell interference. We consider imperfect channel state information (CSI) at the BS, i.e., imperfect CSIT, and focus on the robust optimization of the BS's instantaneous CSI-adaptive beamforming and the IRS's quasi-static phase shifts. First, we formulate the robust optimization of the BS's instantaneous channel state information (CSI)-adaptive beamforming and IRS's quasi-static phase shifts for the ergodic rate maximization as a very challenging two-timescale stochastic non-convex problem. Then, we obtain a closed-form beamformer for any given phase shifts and a more tractable single-timescale stochastic non-convex problem only for phase shifts. Next, we propose a low-complexity stochastic algorithm to obtain quasi-static phase shifts which correspond to a KKT point of the single-timescale stochastic problem. It is worth noting that the proposed method offers a closed-form robust instantaneous CSI-adaptive beamforming design that can promptly adapt to rapid CSI changes over slots and a robust quasi-static phase shift design of low computation and phase adjustment costs in the presence of channel estimation errors and inter-cell interference. Finally, numerical results demonstrate the notable gains of the proposed robust joint design over existing ones and reveal the practical values of the proposed solutions.