Symmetry is an important aesthetic criteria in graph drawing and network visualisation. Symmetric graph drawings aim to faithfully represent automorphisms of graphs as geometric symmetries in a drawing. In this paper, we design and implement a framework for quality metrics that measure symmetry, that is, how faithfully a drawing of a graph displays automorphisms as geometric symmetries. The quality metrics are based on geometry (i.e. Euclidean distance) as well as mathematical group theory (i.e. orbits of automorphisms). More specifically, we define two varieties of symmetry quality metrics: (1) for displaying a single automorphism as a symmetry (axial or rotational) and (2) for displaying a group of automorphisms (cyclic or dihedral). We also present algorithms to compute the symmetric quality metrics in O(n log n) time for rotational symmetry and axial symmetry. We validate our symmetry quality metrics using deformation experiments. We then use the metrics to evaluate a number of established graph drawing layouts to compare how faithfully they display automorphisms of a graph as geometric symmetries.