Building management systems tout numerous benefits, such as energy efficiency and occupant comfort but rely on vast amounts of data from various sensors. Advancements in machine learning algorithms make it possible to extract personal information about occupants and their activities beyond the intended design of a non-intrusive sensor. However, occupants are not informed of data collection and possess different privacy preferences and thresholds for privacy loss. While privacy perceptions and preferences are most understood in smart homes, limited studies have evaluated these factors in smart office buildings, where there are more users and different privacy risks. To better understand occupants’ perceptions and privacy preferences, we conducted twenty-four semi-structured interviews between April 2022 and May 2022 on occupants of a smart office building. We found that data modality features and personal features contribute to people’s privacy preferences. The features of the collected modality define data modality features – spatial, security, and temporal context. In contrast, personal features consist of one’s awareness of data modality features and data inferences, definitions of privacy and security, and the available rewards and utility. Our proposed model of people’s privacy preferences in smart office buildings helps design more effective measures to improve people’s privacy.