Augmenting a cardiology-patient doctor-dialogue through integrated heartbeat-activated holographic display

The causes and treatment solutions of congenital heart defects are difficult to address and discuss between patient and doctor. This is mainly due to the complex spatial nature of congenital cardiac defects, which makes it difficult for the patients to envision the defect without prior anatomical knowledge and renders the comprehension largely dependent on doctors' (variable) skills to describe the anomaly. To improve communication, 3D printed hearts have been developed, yet these are expensive, difficult to manage for the large collection of defects, and require substantial oral explanation. In addition, the correlation with cardiac function remains rather abstract. Instead, we propose an augmented reality solution, involving a see-through head-mounted display (HMD) extended with a built-in heart rate monitor. In order to increase the presence and the conversational power, the heartbeat of the patient is used to drive an animation of a supersized, floating heart visualisation; enabling the user to inspect a specific heart condition from all sides. To enable this, a universal add-on casing was developed for the HoloLens. Heuristic analysis and pilot tests with 6+15 participants reveal limitations of the implementation and show that the solution does increase comprehension, although more has to be done to enable a robust system.