By exploiting the well-known observation that size-biasing or zero-biasing an infinitely divisible random variable may be achieved by adding an independent increment, combined with tools from Stein's method for compound Poisson and Gaussian approximations, we establish three sets of approximation results: (a) bounds on the proximity of Poisson mixtures with infinitely divisible mixing distributions, (b) central limit theorems with explicit error bounds for sums of associated or negatively associated random variables which do not require boundedness of the underlying distributions, and (c) a Gaussian approximation theorem under a vanishing third moment condition. These exploit biasing by an independent increment directly, via an intermediate compound Poisson approximation, and through a convex ordering argument, respectively. Applications include a Dickman-type limit theorem, simple random sampling and urn models with overflow.