A bipartite graph $G=(A, B, E)$ is said to be a biconvex bipartite graph if there exist orderings $<_A$ in $A$ and $<_B$ in $B$ such that the neighbors of every vertex in $A$ are consecutive with respect to $<_B$ and the neighbors of every vertex in $B$ are consecutive with respect to $<_A$. A caterpillar is a tree that will result in a path upon deletion of all the leaves. In this note, we prove that there exists a spanning caterpillar in any connected biconvex bipartite graph. Besides being interesting on its own, this structural result has other consequences. For instance, this directly resolves the burning number conjecture for biconvex bipartite graphs.