It has been noted anecdotally and through a small number of formal studies that ambient lighting conditions and dynamic real-world backgrounds affect the usability of optical see-through augmented reality (AR) displays; especially so in outdoor environments. Our previous work examined these effects using painted posters as representative real-world backgrounds. In this paper, we present a study that employs an experimental testbed that allows AR graphics to be overlaid onto real-world backgrounds as well as painted posters. Our results indicate that color blending effects of physical materials as backgrounds are nearly the same as their corresponding poster backgrounds, even though the colors of each pair are only a metameric match. More importantly, our results suggest that given the current capabilities of optical see-through head-mounted displays (oHMDs), the implications are, at a minimum, a reduced color gamut available to user interface (UI) designers. In worse cases, there are unknown or unexpected color interactions that no UI or system designers can plan for; significantly crippling the usability of the UI or altering the semantic interpretation of graphical elements. Further, our results support the concept of an adaptive AR system which can dynamically alter the color of UI elements based on predicted background color interactions. These interactions can be studied and predicted through methods such as those presented in this work.