Statistical inference in competing risks models is often based on the famous Aalen-Johansen estimator. Since the corresponding limit process lacks independent increments, it is typically applied together with Lin's (1997) resampling technique involving standard normal multipliers. Recently, it has been seen that this approach can be interpreted as a wild bootstrap technique and that other multipliers, as e.g. centered Poissons, may lead to better finite sample performances, see Beyersmann et al. (2013). Since the latter is closely related to Efron's classical bootstrap, the question arises whether this or more general weighted bootstrap versions of Aalen-Johansen processes lead to valid results. Here we analyze their asymptotic behaviour and it turns out that such weighted bootstrap versions in general possess the wrong covariance structure in the limit. However, we explain that the weighted bootstrap can nevertheless be applied for specific null hypotheses of interest and also discuss its limitations for statistical inference. To this end, we introduce different consistent weighted bootstrap tests for the null hypothesis of stochastically ordered cumulative incidence functions and compare their finite sample performance in a simulation study.