A matrix-valued inner product was proposed before to construct orthonormal matrix-valued wavelets for matrix-valued signals. It introduces a weaker orthogonality for matrix-valued signals than the orthogonality of all components in a matrix that is commonly used in orthogonal multiwavelet constructions. With the weaker orthogonality, it is easier to construct orthonormal matrix-valued wavelets. In this paper, we re-study the matrix-valued inner product more from the inner product viewpoint that is more fundamental and propose a new but equivalent norm for matrix-valued signals. We show that although it is not scalar-valued, it maintains most of the scalar-valued inner product properties. We introduce a new linear independence concept for matrix-valued signals and present some related properties. We then present the Gram-Schmidt orthonormalization procedure for a set of linearly independent matrix-valued signals. Finally we define matrix-valued lattices.