Bicyclic codes are a generalization of the one dimensional (1D) cyclic codes to two dimensions (2D). Similar to the 1D case, in some cases, 2D cyclic codes can also be constructed to guarantee a specified minimum distance. Many aspects of these codes are yet unexplored. Motivated by the problem of constructing quantum codes, in this paper, we study some structural properties of certain bicyclic codes. We show that a primitive narrow-sense bicyclic hyperbolic code of length $n^2$ contains its dual if and only if its design distance is lower than $n-Δ$, where $Δ=\mathcal{O}(\sqrt{n})$. We extend the sufficiency condition to the non-primitive case as well. We also show that over quadratic extension fields, a primitive bicyclic hyperbolic code of length $n^2$ contains Hermitian dual if and only if its design distance is lower than $n-Δ_h$, where $Δ_h=\mathcal{O}(\sqrt{n})$. Our results are analogous to some structural results known for BCH and Reed-Solomon codes. They further our understanding of bicyclic codes. We also give an application of these results by showing that we can construct two classes of quantum bicyclic codes based on our results.