Balance theory explains the forces behind the structure of social systems, which are commonly modeled as static undirected signed networks. We expand this modeling approach to incorporate directionality of edges, and consider three levels of analysis: triads, subgroups, and the whole network. For triad-level balance, we operationalize a new measure by utilizing semicycles that satisfy the condition of transitivity. For subgroup-level balance, we propose measures of cohesiveness (intra-group solidarity) and divisiveness (inter-group antagonism) to capture balance within and among subgroups of the network using the most fitting partition of nodes into two groups. For network-level balance, we re-purpose the normalized line index to incorporate directionality, and provide the proportion of edges whose position suits balance. Through extensive computational analysis, we quantify and analyze patterns of social structure in triads, subgroups, and the whole network across a range of social settings from college students and Wikipedia editors to philosophers and Bitcoin traders. We then apply our multilevel framework of analysis to examine balance in temporal and multilayer networks, which demonstrates the generalizability of our approach to evaluating balance, and leads to new observations on balance with respect to time and layer dimensions. Our complementary findings on a variety of social networks highlight the need to evaluate balance at different levels. We propose a comprehensive yet parsimonious approach to address this need.