Here we investigate single-cell stochastic gene expression kinetics in a minimal coupled gene circuit with positive-plus-negative feedback. A triphasic stochastic bifurcation upon the increasing ratio of the positive and negative feedback strengths is observed, which reveals a strong synergistic interaction between positive and negative feedback loops. We discover that coupled positive-plus-negative feedback amplifies gene expression mean but reduces gene expression noise over a wide range of feedback strengths when promoter switching is relatively slow, stabilizing gene expression around a relatively high level. In addition, we study two types of macroscopic limits of the discrete chemical master equation model: the Kurtz limit applies to proteins with large burst frequencies and the Lévy limit applies to proteins with large burst sizes. We derive the analytic steady-state distributions of the protein abundance in a coupled gene circuit for both the discrete model and its two macroscopic limits, generalizing the results obtained in [Chaos 26:043108, 2016]. We also obtain the analytic time-dependent protein distribution for the classical Friedman-Cai-Xie random bursting model proposed in [Phys. Rev. Lett. 97:168302, 2006]. Our analytic results are further applied to study the structure of gene expression noise in a coupled gene circuit and a complete decomposition of noise in terms of five different biophysical origins is provided.