A signed graph is a pair $(G,τ)$ of a graph $G$ and its sign $τ$, where a \textit{sign} $τ$ is a function from $\{ (e,v)\mid e\in E(G),v\in V(G), v\in e\}$ to $\{1,-1\}$. Note that graphs or digraphs are special cases of signed graphs. In this paper, we study the toric ideal $I_{(G,τ)}$ associated with a signed graph $(G,τ)$, and the results of the paper give a unified idea to explain some known results on the toric ideals of a graph or a digraph. We characterize all primitive binomials of $I_{(G,τ)}$, and then focus on the complete intersection property. More precisely, we find a complete list of graphs $G$ such that $I_{(G,τ)}$ is a complete intersection for every sign $τ$.