Hurwitz numbers enumerate branched morphisms between Riemann surfaces. For a fixed elliptic target, Hurwitz numbers are intimately related to mirror symmetry following work of Dijkgraaf. In recent work of Chapuy and Dolega a new variant of Hurwitz numbers with fixed genus $0$ target was introduced that includes maps between between non-orientiable surfaces. These numbers are called $b$-Hurwitz numbers and are polynomials in a parameter $b$ which measures the non-orientability of the involved maps. An interpretation in terms of factorisations of $b$-Hurwitz numbers for $b=1$, so-called twisted Hurwitz numbers, was found in work of Burman and Fesler. In previous work, the authors derived a tropical geometry interpretation of these numbers. In this paper, we introduce a natural generalisation of twisted Hurwitz numbers with elliptic targets within the framework of symmetric groups. We derive a tropical interpretation of these invariants, relate them to Feynman integrals and derive an expression as a matrix element of an operator in the bosonic Fock space.