Consider a birth and death process started from one individual in which each individual gives birth at rate $λ$ and dies at rate $μ$, so that the population size grows at rate $r = λ- μ$. Lambert and Harris, Johnston, and Roberts came up with methods for constructing the exact genealogy of a sample of size $n$ taken from this population at time $T$. We use the construction of Lambert, which is based on the coalescent point process, to obtain asymptotic results for the site frequency spectrum associated with this sample. In the supercritical case $r > 0$, our results extend results of Durrett for exponentially growing populations. In the critical case $r = 0$, our results parallel those that Dahmer and Kersting obtained for Kingman's coalescent.