Multimodal visualization with interactive closeups

Abstract

Closeups are used in illustrations to provide detailed views on regions of interest. They are integrated into the rendering of the whole structure in order to reveal their spatial context. In this paper we present the concept of interactive closeups for medical reporting. Each closeup is associated with a region of interest and may show a single modality or a desired combination of the available modalities using different visualization styles. Thus it becomes possible to visualize multiple modalities simultaneously and to support doctor-to-doctor communication on the basis of interactive multimodal closeup visualizations. We discuss how to compute a layout for 2D and 3D closeups, and how to edit a closeup configuration to prepare a presentation or a subsequent doctor-to-doctor communication. Furthermore, we introduce a GPU-based rendering algorithm, which allows to render multiple closeups at interactive frame rates. We demonstrate the application of the introduced concepts to multimodal PET/CT data sets additionally co-registered with MRI.