Representing various networked data as multiplex networks, networks of networks and other multilayer networks can reveal completely new types of structures in these system. We introduce a general and principled graphlet framework for multilayer networks which allows one to break any multilayer network into small multilayered building blocks. These multilayer graphlets can be either analyzed themselves or used to do tasks such as comparing different systems. The method is flexible in terms of multilayer isomorphism, automorphism orbit definition, and the type of multilayer network. We illustrate our method for multiplex networks and show how it can be used to distinguish networks produced with multiple models from each other in an unsupervised way. In addition, we include an automatic way of generating the hundreds of dependency equations between the orbit counts needed to remove redundant orbit counts. The framework introduced here allows one to analyze multilayer networks with versatile semantics, and these methods can thus be used to analyze the structural building blocks of myriad multilayer networks.