Schur complement matrices emerge in many domain decomposition methods that can solve complex engineering problems using supercomputers. Today, as most of the high-performance clusters' performance lies in GPUs, these methods should also be accelerated. Typically, the offloaded components are the explicitly assembled dense Schur complement matrices used later in the iterative solver for multiplication with a vector. As the explicit assembly is expensive, it represents a significant overhead associated with this approach to acceleration. It has already been shown that the overhead can be minimized by assembling the Schur complements directly on the GPU. This paper shows that the GPU assembly can be further improved by wisely utilizing the sparsity of the input matrices. In the context of FETI methods, we achieved a speedup of 5.1 in the GPU section of the code and 3.3 for the whole assembly, making the acceleration beneficial from as few as 10 iterations.