RealityReplay: Detecting and Replaying Temporal Changes In Situ using Mixed Reality

Humans easily miss events in their surroundings due to limited short-term memory and field of view.

Overview

Authors: Hyunsung Cho, Matthew Komar, David Lindlbauer

Publication Date: 1 January 2023

Link: https://augmented-perception.org/publications/2023-realityreplay.html

Keywords: AR / VR / XR, Context-Aware Computing, time-traveling, egocentric, semantic segmentation, Mixed-Reality (MR), RealityReplay, In-situations Visualization, short-term memory, 360-degree Camera, head-mounted display (HMD), augmented awareness

Type: Peer-Reviewed Journals/White Papers

Summary

Humans easily miss events in their surroundings due to limited short-term memory and field of view. This happens, for example, while watching an instructor's machine repair demonstration or conversing during a sports game. We present RealityReplay, a novel Mixed Reality (MR) approach that tracks and visualizes these significant events using in-situ MR visualizations without modifying the physical space. It requires only a head-mounted MR display and a 360-degree camera.