Internet of Things (IoT) aims at providing connectivity between every computing entity. However, this facilitation is also leading to more cyber threats which may exploit the presence of a vulnerability of a period of time. One such vulnerability is the zero-day threat that may lead to zero-day attacks which are detrimental to an enterprise as well as the network security. In this article, a study is presented on the zero-day threats for IoT networks and a context graph-based framework is presented to provide a strategy for mitigating these attacks. The proposed approach uses a distributed diagnosis system for classifying the context at the central service provider as well as at the local user site. Once a potential zero-day attack is identified, a critical data sharing protocol is used to transmit alert messages and reestablish the trust between the network entities and the IoT devices. The results show that the distributed approach is capable of mitigating the zero-day threats efficiently with 33% and 21% improvements in terms of cost of operation and communication overheads, respectively, in comparison with the centralized diagnosis system.