Probabilistic metric embedding into trees is a powerful technique for designing online algorithms. The standard approach is to embed the entire underlying metric into a tree metric and then solve the problem on the latter. The overhead in the competitive ratio depends on the expected distortion of the embedding, which is logarithmic in $n$, the size of the underlying metric. For many online applications, such as online network design problems, it is natural to ask if it is possible to construct such embeddings in an online fashion such that the distortion would be a polylogarithmic function of $k$, the number of terminals. Our first main contribution is answering this question negatively, exhibiting a \emph{lower bound} of $\tildeΩ(\log k \log Φ)$, where $Φ$ is the aspect ratio of the set of terminals, showing that a simple modification of the probabilistic embedding into trees of Bartal (FOCS 1996), which has expected distortion of $O(\log k \log Φ)$, is \emph{nearly-tight}. Unfortunately, this may result in a very bad dependence in terms of $k$, namely, a power of $k$. Our second main contribution is a general framework for bypassing this limitation. We show that for a large class of online problems this online probabilistic embedding can still be used to devise an algorithm with $O(\min\{\log k\log (kλ),\log^3 k\})$ overhead in the competitive ratio, where $k$ is the current number of terminals, and $λ$ is a measure of subadditivity of the cost function, which is at most $r$, the current number of requests. In particular, this implies the first algorithms with competitive ratio $\operatorname{polylog}(k)$ for online subadditive network design (buy-at-bulk network design being a special case), and $\operatorname{polylog}(k,r)$ for online group Steiner forest.