Let $S$ be a non-empty finite set. A flag of $S$ is a set $f$ of non-empty proper subsets of $S$ such that $X\subseteq Y$ or $Y\subseteq X$ for all $X,Y\in f$. The set $\{|X|:X\in f\}$ is called the type of $f$. Two flags $f$ and $f'$ are in general position with respect to $S$ if $X\cap Y=\emptyset$ or $X\cup Y=S$ for all $X\in f$ and $Y\in f'$. For a fixed type $T$, Klaus Metsch defined the general position graph $Γ(S,T)$ whose vertices are the flags of $S$ of type $T$ with two vertices being adjacent when the corresponding flags are in general position. In this paper, we characterize the full automorphism groups of $Γ(S,T)$ in the case that $|T|=2$. In particular, we solve an open problem proposed by Klaus Metsch.