We focus on a family of one-dimensional probabilistic cellular automata with memory two: the dynamics is such that the value of a given cell at time $t+1$ is drawn according to a distribution which is a function of the states of its two nearest neighbours at time $t$, and of its own state at time $t-1$. Such PCA naturally arise in the study of some models coming from statistical physics ($8$-vertex model, directed animals and gaz models, TASEP, etc.). We give conditions for which the invariant measure has a product form or a Markovian form, and we prove an ergodicity result holding in that context. The stationary space-time diagrams of these PCA present different forms of reversibility. We describe and study extensively this phenomenon, which provides families of Gibbs random fields on the square lattice having nice geometric and combinatorial properties.