In 1991, Roth introduced a natural generalization of rank metric codes, namely tensor codes. The latter are defined to be subspaces of $r$-tensors where the ambient space is endowed with the tensor rank as a distance function. In this work, we describe the general class of tensor codes and we study their invariants that correspond to different families of anticodes. In our context, an anticode is a perfect space that has some additional properties. A perfect space is one that is spanned by tensors of rank 1. Our use of the anticode concept is motivated by an interest in capturing structural properties of tensor codes. In particular, we indentify four different classes of tensor anticodes and show how these gives different information on the codes they describe. We also define the generalized tensor binomial moments and the generalized tensor weight distribution of a code and establish a bijection between these invariants. We use the generalized tensor binomial moments to define the concept of an $i$-tensor BMD code, which is an extremal code in relation to an inequality arising from them. Finally, we give MacWilliams identities for generalized tensor binomial moments.