Sampling from the lattice Gaussian distribution has emerged as an important problem in coding, decoding and cryptography. In this paper, lattice reduction technique is adopted to Gibbs sampler for lattice Gaussian sampling. Firstly, with respect to lattice Gaussian distribution, the convergence rate of systematic scan Gibbs sampling is derived and we show it is characterized by the Hirschfeld-Gebelein-Rényi (HGR) maximal correlation among the multivariate of being sampled. Therefore, lattice reduction is applied to formulate an equivalent lattice Gaussian distribution but with less correlated multivariate, which leads to a better Markov mixing due to the enhanced convergence rate. Then, we extend the proposed lattice-reduction-aided Gibbs sampling to lattice decoding, where the choice of the standard deviation for the sampling is fully investigated. A customized solution that suits for each specific decoding case by Euclidean distance is given, thus resulting in a better trade-off between Markov mixing and sampler decoding. Moreover, based on it, a startup mechanism is also proposed for Gibbs sampler decoding, where decoding complexity can be reduced without performance loss. Simulation results based on large-scale MIMO detection are presented to confirm the performance gain and complexity reduction.