Human/User interaction with buildings are mostly restricted to interacting with building automation systems through user-interfaces that mainly aim to improve energy efficiency of buildings and ensure comfort of occupants. This research builds on the existing theories of Human-Building Interaction (HBI) and proposes a novel conceptual framework for HBI that combines the concepts of Human-Computer Interaction (HCI) and Ambient Intelligence (AmI). The proposed framework aims to study the needs of occupants in specific-purpose buildings, which is currently undermined. Specifically, we explore the application of the proposed HBI framework to improve the learning experience of students in academic buildings. Focus groups and semi-structured interviews were conducted among students who are considered primary occupants of Goodwin Hall, a flagship smart engineering building at Virginia Tech. Qualitative coding and concept mapping were used to analyze the qualitative data and determine the impact of occupant-specific needs on the learning experience of students in academic buildings. The occupant-specific problem that was found to have the highest direct impact on learning experience was finding study space and highest indirect impact was Indoor Environment Quality (IEQ). We discuss new ideas for designing Intelligent User Interfaces (IUI), e.g. Augmented Reality (AR), increase the perceivable affordances for building occupants and considering a context-aware ubiquitous analytics-based strategy to provide services that are tailored to address the identified needs.