Ensuring the trustworthiness of graph neural networks (GNNs), which are often treated as black-box models, requires effective explanation techniques. Existing GNN explanations typically apply input perturbations to identify subgraphs that are responsible for the occurrence of the final output of GNNs. However, such approaches lack finer-grained, layer-wise analysis of how intermediate representations contribute to the final result, capabilities that are crucial for model diagnosis and architecture optimization. This paper introduces SliceGX, a novel GNN explanation approach that generates explanations at specific GNN layers in a progressive manner. Given a GNN model M, a set of selected intermediate layers, and a target layer, SliceGX slices M into layer blocks("model slice") and discovers high-quality explanatory subgraphs within each block that elucidate how the model output arises at the target layer. Although finding such layer-wise explanations is computationally challenging, we develop efficient algorithms and optimization techniques that incrementally construct and maintain these subgraphs with provable approximation guarantees. Extensive experiments on synthetic and real-world benchmarks demonstrate the effectiveness and efficiency of SliceGX, and illustrate its practical utility in supporting model debugging.