This work is a follow-up to our previous contribution ("Convergence of sparse collocation for functions of countably many Gaussian random variables (with application to elliptic PDEs)", SIAM J. Numer. Anal., 2018), and contains further insights on some aspects of the solution of elliptic PDEs with lognormal diffusion coefficients using sparse grids. Specifically, we first focus on the choice of univariate interpolation rules, advocating the use of Gaussian Leja points as introduced by Narayan and Jakeman ("Adaptive Leja sparse grid constructions for stochastic collocation and high-dimensional approximation", SIAM J. Sci. Comput., 2014) and then discuss the possible computational advantages of replacing the standard Karhunen-Loève expansion of the diffusion coefficient with the Lévy-Ciesielski expansion, motivated by theoretical work of Bachmayr, Cohen, DeVore, and Migliorati ("Sparse polynomial approximation of parametric elliptic PDEs. part II: lognormal coefficients", ESAIM: M2AN, 2016). Our numerical results indicate that, for the problem under consideration, Gaussian Leja collocation points outperform Gauss-Hermite and Genz-Keister nodes for the sparse grid approximation and that the Karhunen-Loève expansion of the log diffusion coefficient is more appropriate than its Lévy-Ciesielski expansion for purpose of sparse grid collocation.