Investigating Search Among Physical and Virtual Objects Under Different Lighting Conditions

Abstract

By situating computer-generated content in the physical world, mobile augmented reality (AR) can support many tasks that involve effective search and inspection of physical environments. Currently, there is limited information regarding the viability of using AR in realistic wide-area outdoor environments and how AR experiences affect human behavior in these environments. Here, we conducted a wide-area outdoor AR user study (n=48) using a commercially available AR headset (Microsoft Hololens 2) to compare (1) user interactions with physical and virtual objects in the environment (2) the effects of different lighting conditions on user behavior and AR experience and (3) the impact of varying cognitive load on AR task performance. Participants engaged in a treasure hunt task where they searched for and classified virtual target items (green “gems”) in an augmented outdoor courtyard scene populated with physical and virtual objects. Cognitive load was manipulated so that in half the search trials users were required to monitor an audio stream and respond to specific target sounds. Walking paths, head orientation and eye gaze information were measured, and users were queried about their memory of encountered objects and provided feedback on the experience. Key findings included (1) Participants self-reported significantly lower comfort in the ambient natural light condition, with virtual objects more visible and participants more likely to walk into physical objects at night; (2) recall for physical objects was worse than for virtual objects, (3) participants discovered more gems hidden behind virtual objects than physical objects, implying higher attention on virtual objects and (4) dual-tasking modified search behavior. These results suggest there are important technical, perceptual and cognitive factors that must be considered if the full potential of “anywhere and anytime mobile AR” is to be realized.