Would-be practitioners of causal discovery face a dizzying array of algorithms without a clear best choice. This abundance of competitive methods makes ensembling a natural strategy for practical applications. At the same time, real-world use cases frequently violate the assumptions on which common causal discovery algorithms are based, forcing reliance on expert knowledge. Inspired by recent work on dynamically requested expert knowledge and large language models (LLMs) as experts, we present a flexible model averaging method that integrates selective expert querying to ensemble a diverse set of causal discovery algorithms. Crucially, we distinguish between edge existence and orientation, enabling the method to leverage the complementary strengths of data-driven discovery and expert input. We further consider the realistic setting of limited access to an imperfect expert, using disagreement among algorithms to query the expert in cases of greater uncertainty. Experiments demonstrate that our method consistently outperforms strong baselines on both clean and noisy data. Code and data are available at https://anonymous.4open.science/r/expert-cd-ensemble-3282/.