In this paper, we consider queue-aware beamforming control for video streaming applications in multi-antenna interference network. Using heavy traffic approximation technique, we first derive the diffusion limit for the discrete time queuing system. Based on the diffusion limit, we formulate an infinite horizon ergodic control problem to minimize the average power costs of the base stations subject to the constraints on the playback interruption costs and buffer overflow costs of the mobile users. To deal with the queue coupling challenge, we utilize the weak interference coupling property in the network to derive a closed-form approximate value function of the optimality equation as well as the associated error bound using perturbation analysis. Based on the closed-form approximate value function, we propose a low complexity queue-aware beamforming control algorithm, which is asymptotically optimal for sufficiently small cross-channel path gain. Finally, the proposed scheme is compared with various baselines through simulations and it is shown that significant performance gain can be achieved.