Let $(X,§)$ be a set system on an $n$-point set $X$. The \emph{discrepancy} of $§$ is defined as the minimum of the largest deviation from an even split, over all subsets of $S \in §$ and two-colorings $χ$ on $X$. We consider the scenario where, for any subset $X' \subseteq X$ of size $m \le n$ and for any parameter $1 \le k \le m$, the number of restrictions of the sets of $§$ to $X'$ of size at most $k$ is only $O(m^{d_1} k^{d-d_1})$, for fixed integers $d > 0$ and $1 \le d_1 \le d$ (this generalizes the standard notion of \emph{bounded primal shatter dimension} when $d_1 = d$). In this case we show that there exists a coloring $χ$ with discrepancy bound $O^{*}(|S|^{1/2 - d_1/(2d)} n^{(d_1 - 1)/(2d)})$, for each $S \in §$, where $O^{*}(\cdot)$ hides a polylogarithmic factor in $n$. This bound is tight up to a polylogarithmic factor \cite{Mat-95, Mat-99} and the corresponding coloring $χ$ can be computed in expected polynomial time using the very recent machinery of Lovett and Meka for constructive discrepancy minimization \cite{LM-12}. Our bound improves and generalizes the bounds obtained from the machinery of Har-Peled and Sharir \cite{HS-11} (and the follow-up work in \cite{SZ-12}) for points and halfspaces in $d$-space for $d \ge 3$.