We study a branching Brownian motion $Z$ in $\mathbb{R}^d$, among obstacles scattered according to a Poisson random measure with a radially decaying intensity. Obstacles are balls with constant radius and each one works as a trap for the whole motion when hit by a particle. Considering a general offspring distribution, we derive the decay rate of the annealed probability that none of the particles of $Z$ hits a trap, asymptotically in time $t$. This proves to be a rich problem motivating the proof of a more general result about the speed of branching Brownian motion conditioned on non-extinction. We provide an appropriate "skeleton" decomposition for the underlying Galton-Watson process when supercritical and show that the "doomed" particles do not contribute to the asymptotic decay rate.