Classical model-based imaging methods for ultrasound elasticity inverse problem require prior constraints about the underlying elasticity patterns, while finding the appropriate hand-crafted prior for each tissue type is a challenge. In contrast, standard data-driven methods count solely on supervised learning on the training data pairs leading to massive network parameters for unnecessary physical model relearning which might not be consistent with the governing physical models of the imaging system. Fusing the physical forward model and noise statistics with data-adaptive priors leads to a united reconstruction framework that guarantees the learned reconstruction agrees with the physical models while coping with the limited training data. In this paper, we propose a new methodology for estimating the elasticity image by solving a regularized optimization problem which benefits from the physics-based modeling via a data-fidelity term and adversarially learned priors via a regularization term. In this method, the regularizer is trained based on the Wasserstein Generative Adversarial Network (WGAN) objective function which tries to distinguish the distribution of clean and noisy images. Leveraging such an adversarial regularizer for parameterizing the distribution of latent images and using gradient descent (GD) for solving the corresponding regularized optimization task leads to stability and convergence of the reconstruction compared to pixel-wise supervised learning schemes. Our simulation results verify the effectiveness and robustness of the proposed methodology with limited training datasets.