With humans interacting with AI-based systems at an increasing rate, it is necessary to ensure the artificial systems are acting in a manner which reflects understanding of the human. In the case of humans and artificial AI agents operating in the same environment, we note the significance of comprehension and response to the actions or capabilities of a human from an agent's perspective, as well as the possibility to delegate decisions either to humans or to agents, depending on who is deemed more suitable at a certain point in time. Such capabilities will ensure an improved responsiveness and utility of the entire human-AI system. To that end, we investigate the use of cognitively inspired models of behavior to predict the behavior of both human and AI agents. The predicted behavior, and associated performance with respect to a certain goal, is used to delegate control between humans and AI agents through the use of an intermediary entity. As we demonstrate, this allows overcoming potential shortcomings of either humans or agents in the pursuit of a goal.