The concept of paraunitary (PU) matrices arose in the early 1990s in the study of multi-rate filter banks. So far, these matrices have found wide applications in cryptography, digital signal processing, and wireless communications. Existing PU matrices are subject to certain constraints on their existence and hence their availability is not guaranteed in practice. Motivated by this, for the first time, we introduce a novel concept, called $Z$-paraunitary (ZPU) matrix, whose orthogonality is defined over a matrix of polynomials with identical degree not necessarily taking the maximum value. We show that there exists an equivalence between a ZPU matrix and a $Z$-complementary code set when the latter is expressed as a matrix with polynomial entries. Furthermore, we investigate some important properties of ZPU matrices, which are useful for the extension of matrix sizes and sequence lengths. Finally, we propose a unifying construction framework for optimal ZPU matrices which includes existing PU matrices as a special case.