Our high expectations from Internet of Things (IoT) and how it will positively influence our lifestyles depend on a secure and trusted implementation of it, especially in the sensitive sectors such as health or financial. IoT platforms and solutions must provide Confidentiality, Integrity and Availability (CIA) in a secure and transparent way. Due to the extremely large scales of IoT, traditional centralized solutions for security provisioning cannot be employed in their original form. This article discusses the authentication problem in IoT, which is fundamental to providing CIA. We propose a hierarchical security architecture for this problem and focus on computationally lightweight authentication protocols which can intelligently distribute the computational load across multiple levels and effectively push the load towards upper layers.