We study conic divisorial ideals from the viewpoint of matroid theory and apply the resulting framework to toric rings arising from signed posets. For a toric ring, we describe the polytope representing divisor classes corresponding to conic divisorial ideals in terms of matroids. We then turn to the toric ring $R_P$ associated with a signed poset $P$. We compute the divisor class group and characterize the ($\mathbb{Q}$-)Gorenstein property of $R_P$ in terms of $P$. Moreover, we also construct a polytope characterizing the conic divisorial ideals of $R_P$. This recovers and extends previous results on Hibi rings to the setting of signed posets.