This paper considers the generalized bilinear recovery problem which aims to jointly recover the vector $\mathbf b$ and the matrix $\mathbf X$ from componentwise nonlinear measurements ${\mathbf Y}\sim p({\mathbf Y}|{\mathbf Z})=\prod\limits_{i,j}p(Y_{ij}|Z_{ij})$, where ${\mathbf Z}={\mathbf A}({\mathbf b}){\mathbf X}$, ${\mathbf A}(\cdot)$ is a known affine linear function of $\mathbf b$, and $p(Y_{ij}|Z_{ij})$ is a scalar conditional distribution which models the general output transform. A wide range of real-world applications, e.g., quantized compressed sensing with matrix uncertainty, blind self-calibration and dictionary learning from nonlinear measurements, one-bit matrix completion, etc., can be cast as the generalized bilinear recovery problem. To address this problem, we propose a novel algorithm called the Bilinear Adaptive Generalized Vector Approximate Message Passing (BAd-GVAMP), which extends the recently proposed Bilinear Adaptive Vector AMP (BAd-VAMP) algorithm to incorporate arbitrary distributions on the output transform. Numerical results on various applications demonstrate the effectiveness of the proposed BAd-GVAMP algorithm.