Inertial sensors are crucial for recognizing pedestrian activity. Recent advances in deep learning have greatly improved inertial sensing performance and robustness. Different domains and platforms use deep-learning techniques to enhance network performance, but there is no common benchmark. The latter is crucial for fair comparison and evaluation within a standardized framework. The aim of this paper is to fill this gap by defining and analyzing ten data-driven techniques for improving neural inertial classification networks. In order to accomplish this, we focused on three aspects of neural networks: network architecture, data augmentation, and data preprocessing. The experiments were conducted across four datasets collected from 78 participants. In total, over 936 minutes of inertial data sampled between 50-200Hz were analyzed. Data augmentation through rotation and multi-head architecture consistently yields the most significant improvements. Additionally, this study outlines benchmarking strategies for enhancing neural inertial classification networks.