One key challenge for solving a general stochastic optimization problem with expectations in the objective and constraint functions using ordinary stochastic iterative methods lies in the infeasibility issue caused by the randomness over iterates. This letter aims to address this main challenge. First, we obtain an equivalent stochastic optimization problem which is to minimize the weighted sum of the original objective and the penalty for violating the original constraints. Then, we propose a stochastic successive convex approximation (SSCA) method to obtain a stationary point of the original stochastic optimization problem. Using similar techniques, we propose a parallel SSCA method to obtain a stationary point of a special case of the general stochastic optimization problem which has decoupled constraint functions. We also provide application examples of the proposed methods in power control for interference networks. The proposed SSCA and parallel SSCA methods achieve empirically higher convergence rates and lower computational complexities than existing ones, benefiting from the elegant way of balancing the objective minimization and constraint satisfaction over random iterates.