Fitch graphs $G=(X,E)$ are di-graphs that are explained by $\{\otimes,1\}$-edge-labeled rooted trees with leaf set $X$: there is an arc $xy\in E$ if and only if the unique path in $T$ that connects the least common ancestor $\textrm{lca}(x,y)$ of $x$ and $y$ with $y$ contains at least one edge with label $1$. In practice, Fitch graphs represent xenology relations, i.e., pairs of genes $x$ and $y$ for which a horizontal gene transfer happened along the path from $\textrm{lca}(x,y)$ to $y$. In this contribution, we generalize the concept of xenology and Fitch graphs and consider complete di-graphs $K_{|X|}$ with vertex set $X$ and a map $ε$ that assigns to each arc $xy$ a unique label $ε(x,y)\in M\cup \{\otimes\}$, where $M$ denotes an arbitrary set of symbols. A di-graph $(K_{|X|},ε)$ is a generalized Fitch graph if there is an $M\cup \{\otimes\}$-edge-labeled tree $(T,λ)$ that can explain $(K_{|X|},ε)$. We provide a simple characterization of generalized Fitch graphs $(K_{|X|},ε)$ and give an $O(|X|^2)$-time algorithm for their recognition as well as for the reconstruction of the unique least resolved phylogenetic tree that explains $(K_{|X|},ε)$.