We show that as $n$ changes, the characteristic polynomial of the $n\times n$ random matrix with i.i.d. complex Gaussian entries can be described recursively through a process analogous to Pólya's urn scheme. As a result, we get a random analytic function in the limit, which is given by a mixture of Gaussian analytic functions. This gives another reason why the zeros of Gaussian analytic functions and the Ginibre ensemble exhibit similar local repulsion, but different global behavior. Our approach gives new explicit formulas for the limiting analytic function.