The theory of compressed sensing tells us that recovering all k-sparse signals requires a sensing matrix to satisfy that its spark is greater than 2k, or its order of the null space property (NSP) or the restricted isometry property (RIP) is 2k or above. If we perform elementary row or column operations on the sensing matrix, what are the changes of its spark, NSP order and RIP order? In this paper, we study this problem and discover that these three quantitative indexes of sensing matrices all possess invariance under all matrix elementary transformations except column-addition ones. Putting this result in form of matrix products, we get the types of matrices which multiply a sensing matrix and make the products still have the same properties of sparse recovery as the sensing matrix. According to these types of matrices, we made an interesting discovery that sensing matrices with deterministic constructions do not possess the property universality which belongs to sensing matrices with random constructions.