Facial tracking and augmentation for personalized in situ learning.

A Magic Mirror for Facial Anatomy Tracking

Significance: Anatomy is one of the most challenging subject matters to both teach and learn, and facial anatomy is one of the most complex anatomical regions. Current education relies heavily on cadaveric dissection, supplemented by textbooks; however, human cadavers are expensive and limited in availability, and reading textbooks can feel like a chore to some students. Augmented reality (AR) has the potential to transform the process of learning human anatomy. By blending real and virtual objects, AR allows learners to immerse themselves in the subject matter and take an active role in the learning process. With this technology, students will interact with physiologically accurate and realistic 3-dimensional (3-D) anatomical structures to enhance their understanding of function, form, and spatial relationships. In addition, the 3-D environments are perceived as more lifelike, more interesting, and more enjoyable, which can increase desire to learn and translate into increased knowledge retention.

Innovation and Approach: We propose the design of an augmented reality simulator for facial anatomy. To create this device, we will first digitize a highly detailed atlas of the human facial cardiovascular and nervous system. Second, we will develop an operating system-independent computer application that uses the webcam video stream to track the users face in real-time and superimpose the facial anatomy map on the user’s own face, mimicking a real-world physical mirror. This simulation device can be integrated into multiple courses currently offered at Johns Hopkins University, including the undergraduate, medical, and nursing schools. We will set up a project homepage with instructions on how to obtain, install, and run the AR learning software, and download links for our system. This tool will allow students to navigate within and around 3-D facial anatomical structures in a completely controlled environment. We hypothesize that students who use our AR environment will find learning more enjoyable and engaging, resulting in higher motivation to study with potential improved learning outcomes. In addition, this “magic mirror” effect (ie. interaction with anatomical structures personalized to the user’s own face) will evoke an emotional response that may have an additional influence on the learning process, for example, a deeper understanding and appreciation of function and structure.