This paper examines the construction of confidence sets for parameters defined as linear functionals of a function of W and X whose conditional mean given Z and X equals the conditional mean of another variable Y given Z and X. Many estimands of interest in causal inference can be expressed in this form, including the average treatment effect in proximal causal inference and treatment effect contrasts in instrumental variable models. We derive a necessary condition for a confidence set to be uniformly valid over a model that allows for the dependence between W and Z given X to be arbitrarily weak. Specifically, we show that for any such confidence set, there must exist some laws in the model under which, with high probability, the confidence set has a diameter greater than or equal to the diameter of the parameter's range. In particular, consistent with the weak instruments literature, Wald confidence intervals are not uniformly valid over the aforementioned model when the parameter's range is infinite. Furthermore, we argue that inverting the score test, a successful approach in that literature, generally fails for the broader class of parameters considered here. We present a method for constructing uniformly valid confidence sets in the special case where all variables, but possibly Y, are binary and discuss its limitations. Finally, we emphasize that developing uniformly valid confidence sets for the class of parameters considered in this paper remains an open problem.