Tests of independence are an important tool in applications, specifically in connection with the detection of a relationship between variables; they also have initiated many developments in statistical theory. In the present paper we build upon and extend a recently established link to Discrete Mathematics and Theoretical Computer Science, exemplified by the appearance of copulas in connection with limits of permutation sequences, and by the connection between quasi-randomness and consistency of pattern-based tests of independence. The latter include classical procedures, such as Kendall's tau, which uses patterns of length two. Longer patterns lead to tests that are consistent against large classes of alternatives, as first shown by Hoeffding (1948) with patterns of length five, and by Yanagimoto (1970) and Bergsma and Dassios (2014) for patterns of length four. More recently Chan et al.\ (2020) characterized quasi-randomness for sets of patterns of length four, which leads to several new consistent pattern-based test for independence. We give a detailed and complete description of the respective limiting null distributions. In connection with the power performance of the tests, which is of interest for practical purposes, we provide results on their (local) asymptotic relative efficiencies. We also include a small simulation study that supports our theoretical findings.