With the growth of internet of things (IoT) devices, cyberattacks, such as distributed denial of service, that exploit vulnerable devices infected with malware have increased. Therefore, vendors and users must keep their device firmware updated to eliminate vulnerabilities and quickly handle unknown cyberattacks. However, it is difficult for both vendors and users to continually keep the devices safe because vendors must provide updates quickly and the users must continuously manage the conditions of all deployed devices. Therefore, to ensure security, it is necessary for a system to adapt autonomously to changes in cyberattacks. In addition, it is important to consider network-side security that detects and filters anomalous traffic at the gateway to comprehensively protect those devices. This paper proposes a self-adaptive anomaly detection system for IoT traffic, including unknown attacks. The proposed system comprises a honeypot server and a gateway. The honeypot server continuously captures traffic and adaptively generates an anomaly detection model using real-time captured traffic. Thereafter, the gateway uses the generated model to detect anomalous traffic. Thus, the proposed system can adapt to unknown attacks to reflect pattern changes in anomalous traffic based on real-time captured traffic. Three experiments were conducted to evaluate the proposed system: a virtual experiment using pre-captured traffic from various regions across the world, a demonstration experiment using real-time captured traffic, and a virtual experiment using a public dataset containing the traffic generated by malware. The experimental results indicate that a system adaptable in real time to evolving cyberattacks is a novel approach for ensuring the comprehensive security of IoT devices against both known and unknown attacks.