Recently a growing literature study a new forecast aggregation setting where each forecaster is additionally asked ``what's your expectation for the average of other forecasters' forecasts?''. However, most theoretic results in this setting focus on the scenarios where the additional second-order information helps optimally aggregate the forecasts. Here we adopt an adversarial approach and follow the robust forecast aggregation framework proposed by Arielia, Babichenkoa, and Smorodinsky 2018. We delicately analyze the possibility/impossibility of the new setting when there are two forecasters that either are refinement-ordered or receive conditionally independent and identically distributed (c.i.i.d.) signals. We also extend the setting to a higher level of expectation setting where we can additionally ask ``what's your expectation for the other forecaster's expectation for ...''. The results show that in the above settings, the additional second-order information can significantly improve the aggregation accuracy, and the higher the order, the higher the improvement.