The multigroup neutron transport equations has been widely used to study the interactions of neutrons with their background materials in nuclear reactors. High-resolution simulations of the multigroup neutron transport equations using modern supercomputers require the development of scalable parallel solving techniques. In this paper, we study a scalable transport method for solving the algebraic system arising from the discretization of the multigroup neutron transport equations. The proposed transport method consists of a fully coupled Newton solver for the generalized eigenvalue problems and GMRES together with a novel multilevel domain decomposition preconditioner for the Jacobian system. The multilevel preconditioner has been successfully used for many problems, but the construction of coarse spaces for certain problems, especially for unstructured mesh problems, is expensive and often unscalable. We introduce a new subspace-based coarsening algorithm to address this issue by exploring the structure of the matrix in the discretized version of the neutron transport problems. We numerically demonstrate that the proposed transport method is highly scalable with more than 10,000 processor cores for the 3D C5G7 benchmark problem on unstructured meshes with billions of unknowns. Compared with the traditional multilevel domain decomposition method, the new approach equipped with the subspace-based coarsening algorithm is much faster on the construction of coarse spaces.