Let $L(λ)$ be a simple highest weight module of a classical Lie algebra $\mathfrak{g}$ with highest weight $λ-ρ$, where $ρ$ is half the sum of positive roots. Joseph proved that the associated variety of the annihilator ideal of $L(λ)$ (also called the annihilator variety) is the Zariski closure of a nilpotent orbit in $\mathfrak{g}^*$. Recently, Bai--Ma--Wang introduced a partition algorithm to describe this corresponding nilpotent orbit for a given highest weight module $L(λ)$. In this paper, we present a new direct proof of Bai--Ma--Wang's partition algorithm using Sommers duality.