SNJL-LiU Reading Session No.2 - "Embodied Multimodal Interaction with a Portable Mixed Reality-based Digital Tower System"

Speaker: Dr. Pallavi Mohan is a Research Fellow of the Saab-NTU Joint Lab in the field of VR, with experience in HCI, Computer Vision, Computer Graphics and 3D animation. (https://www.linkedin.com/in/pallavi-mohan/) 

With recent advances in mixed reality (MR) systems and rendering, it is increasingly possible to replicate a complex real-world system into a high-fidelity holographic representation in mixed reality. Air traffic control, a spatially complex system that facilitates safety-critical operations by air traffic controllers (ATCOs), is one pertinent area that benefits from such a digital representation. Research in the past decade has focused on Digital Towers (DTs) that replace physical towers by reproducing live camera feeds of Out-Of-Tower (OOT) views using immersive screens in a CAVE-like set-up that is not co-located with the physical tower. While DTs have made it convenient for multiple aerodromes to be controlled from a single, remote location, they still require the use of multiple expensive spatial displays that span entire walls and even rooms. With the current COVID situation causing a sharp decline in air traffic, the quest to introduce cost-effective alternative solutions for DTs seems more urgent than ever. Nonetheless, the ATCO community has been hesitant to adopt such digital remote set-ups due to the unfamiliarity of the environment. Recent developments in MR-based visualizations have allowed complex spatial data to be better represented in a stereoscopic virtual space. Moreover, the increasingly robust technical capabilities of self-contained MR-based Head-Mounted Devices (HMDs) (such as Microsoft HoloLens 2) gives rise to the possibility of a cohesive DT system that is cost-effective, portable and interactive. Therefore, we propose an interactive MR-based DT system that aims to be a viable alternative to the present immersive DT environments.