Large commercial buildings are complex cyber-physical systems containing expensive and critical equipment that ensure the safety and comfort of their numerous occupants. Yet occupant and visitor access to spaces and equipment within these buildings are still managed through unsystematic, inefficient, and human-intensive processes. As a standard practice, long-term building occupants are given access privileges to rooms and equipment based on their organizational roles, while visitors have to be escorted by their hosts. This approach is conservative and inflexible. In this paper, we describe a methodology that can flexibly and securely manage building access privileges for long-term occupants and short-term visitors alike, taking into account the risk associated with accessing each space within the building. Our methodology relies on blockchain smart contracts to describe, grant, audit, and revoke fine-grained permissions for building occupants and visitors, in a decentralized fashion. The smart contracts are specified through a process that leverages the information compiled from Brick and BOT models of the building. We illustrate the proposed method through a typical application scenario in the context of a real office building and argue that it can greatly reduce the administration overhead, while, at the same time, providing fine-grained, auditable access control.