We show that plane bipolar posets (i.e., plane bipolar orientations with no transitive edge) and transversal structures can be set in correspondence to certain (weighted) models of quadrant walks, via suitable specializations of a bijection due to Kenyon, Miller, Sheffield and Wilson. We then derive exact and asymptotic counting results. In particular we prove (computationally and then bijectively) that the number of plane bipolar posets on $n+2$ vertices equals the number of plane permutations of size $n$. Regarding transversal structures, for each $v\geq 0$ we consider $t_n(v)$ the number of such structures with $n+4$ vertices and weight $v$ per quadrangular inner face (the case $v=0$ corresponds to having only triangular inner faces). We obtain a recurrence to compute $t_n(v)$, and an asymptotic formula that for $v=0$ gives $t_n(0)\sim c\ \!(27/2)^nn^{-1-π/\mathrm{arccos}(7/8)}$ for some $c>0$, which also ensures that the associated generating function is not D-finite.