We present the first efficient averaging sampler that achieves asymptotically optimal randomness complexity and near-optimal sample complexity. For any $δ< \varepsilon$ and any constant $α> 0$, our sampler uses $m + O(\log (1 / δ))$ random bits to output $t = O((\frac{1}{\varepsilon^2} \log \frac{1}δ)^{1 + α})$ samples $Z_1, \dots, Z_t \in \{0, 1\}^m$ such that for any function $f: \{0, 1\}^m \to [0, 1]$, \[ \Pr\left[\left|\frac{1}{t}\sum_{i=1}^t f(Z_i) - \mathbb{E}[f]\right| \leq \varepsilon\right] \geq 1 - δ. \] The randomness complexity is optimal up to a constant factor, and the sample complexity is optimal up to the $O((\frac{1}{\varepsilon^2} \log \frac{1}δ)^α)$ factor. Our technique generalizes to matrix samplers. A matrix sampler is defined similarly, except that $f: \{0, 1\}^m \to \mathbb{C}^{d \times d}$ and the absolute value is replaced by the spectral norm. Our matrix sampler achieves randomness complexity $m + \widetilde O (\log(d / δ))$ and sample complexity $ O((\frac{1}{\varepsilon^2} \log \frac{d}δ)^{1 + α})$ for any constant $α> 0$, both near-optimal with only a logarithmic factor in randomness complexity and an additional $α$ exponent on the sample complexity. We use known connections with randomness extractors and list-decodable codes to give applications to these objects. Specifically, we give the first extractor construction with optimal seed length up to an arbitrarily small constant factor above 1, when the min-entropy $k = βn$ for a large enough constant $β< 1$. Finally, we generalize the definition of averaging sampler to any normed vector space.