Hirschsprung Disease is characterized by the absence of ganglion cells in the myenteric plexus. Therefore, the correct identification of ganglion cells is crucial for diagnosing Hirschsprung disease. We introduce a three-stage analysis framework that mimics the pathologist's diagnostic approach. The framework, based on a Vision Transformer model (ViT-B/16), sequentially segments the muscularis propria, segments the myenteric plexus, and detects ganglion cells within anatomically valid regions. 30 whole-slide images of colon tissue were used, each containing manual annotations of muscularis, plexus, and ganglion cells. A 5-fold cross-validation scheme was applied to each stage, along with resolution-specific tiling strategies and tailored postprocessing to ensure anatomical consistency. The proposed method achieved a Dice coefficient of 89.9% and a Plexus Inclusion Rate of 100% for muscularis segmentation. Plexus segmentation reached a recall of 94.8%, a precision of 84.2% and a Ganglia Inclusion Rate of 99.7%. For ganglion cells annotated with high certainty, the model achieved 62.1\% precision and 89.1% recall. When considering all annotated ganglion cells, regardless of certainty level, the overall precision was 67.0%. These results indicate that ViT-based models are effective at leveraging global tissue context and capturing cellular morphology at small scales, even within complex histological tissue structures. This multi-stage methodology has great potential to support digital pathology workflows by reducing inter-observer variability and assisting in the evaluation of Hirschsprung disease. The clinical impact will be evaluated in future work with larger multi-center datasets and additional expert annotations.