Graphics & Geometry Group

CITmed: Cognitive Interaction Technology for Medical Applications

The CITmed project aims at the development of a novel medical device for the application of virtual reality (VR) in the diagnosis and rehabilitation in neurology, neuropsychology, and psychiatry, as well as the implementation of respective clinical studies. The fields of application will be primarily disturbances of brain functions resulting from apoplexia, cerebral trauma caused by accidents, and neurological and psychiatric diseases.

Despite the huge social costs resulting from such disease patterns, methods for diagnosis and, in particular, rehabilitative treatment of patients are still insufficient. This is particularly evident in cases where higher cognitive functions are affected. Recent experimental studies have shown that VR techniques can build up dynamic, interactive stimulus-environments that are applicable for (i) the diagnosis of behavioral reactions resulting from cognitive processing, and (ii) the installation of interactions between humans and VR environments allowing for the rehabilitation of disturbed brain functions in a context close to everyday life.

The main objective of the CITmed project is to provide a novel systems-platform for the application of interactive VR diagnosis and rehabilitation in neurology, neuropsychology, and psychiatry. We are developing a cost-efficient and low maintenance VR system for the easy-to-use interaction of patients with virtual everyday scenarios. The diagnosis and rehabilitation will be based on novel neuropsychological tests that are custom-tailored to our VR environment. The software to be developed targets clinical use. clinical study.

As a winner of the HighTech.NRW competition, the CITmed project is funded through the NRW Ziel 2 program EFRE (Europäischer Fonds für regionale Entwicklung). CITmed is embedded into Bielefeld's Center of Excellence CITEC, which is the ideal environment for this highly interdisciplinary project.

CITmed News

citmed 01.09.2009: Official start of the CITmed project!
medica 16.11.2010: The CITmed project is exhibited at the MEDICA 2010 trade fair (see the official
westfalen-blatt 03.12.2010: Westfalen-Blatt wrote a nice article about the CITmed project.
CITEC Newsletter 05.08.2011: The CITmed project was featured in the CITEC Newsletter .
Eurographics 06.05.2013: The CITmed project was awarded with the 3rd place of the Eurographics Medical Prize.
UniBi 05.07.2013: The CITmed project was featured in a press release of Bielefeld University.
Gehirn und Geist 10.07.2013: Short article about the CITmed project in Gehirn und Geist.
Item24 16.07.2013: CITmed project was featured in an item-Anwenderbericht.
compamed 11.09.2013: Interview about the CITmed project on Compamed.
WDR Lokalzeit OWL 07.11.2013: The CITmed projekt was featured in WDR Lokalzeit.

CITmed in a Nutshell

While there exist many established rehabilitation programs for patients with disturbed brain functions and cognitive deficits, they all suffer from one major problem: Patients improve their cognitive capacities in the employed paper-and-pencil tests, but this improvement cannot be transferred to daily situations and problems. A VR training system has the advantage of being much more flexible than standard paper-and-pencil tests. We therefore try to overcome the above limitations by designing as realistic as possible VR training scenarios that are close to every day routines. One example is shopping in a virtual supermarket, which requires (and hence trains!) memory, spatial attention, spatial orientation, and executive functions.

The main technical challenges for designing a successfull VR platform for clinical use are cost efficiency, a low level of maintenance, ease of use for the patient, and a sufficiently high level of immersion. In order to meet these goals, our OctaVis system, described in [8], builds a 360° projection by arranging eight screens in an octagon around the patient (Figure 1).

Figure 1: The OctaVis system consists of a ring of 8 monitors, providing a 360-degree projection.

A major challenge is the design of an easy-to-use interaction metaphor for navigating through and interacting with the virtual environment. In the OctaVis system the user navigates by rotating the chair into the direction of desired movement and using a “throttle joystick” in the armrest. Items can be selected by simply touching them on the touch-sensitive screens. This user interaction, depicted in Figure 2 and shown the video below, has been successfully evaluated, such that even elderly patients without previous experience in virtual reality could work with our system [3,8].

Figure 2: User interaction in the OctaVis system.

In contrast to sophisticated and expensive CAVE solutions, which employ a network-connected rendering machine per display, our system is driven by a single PC workstation. Equipped with three graphics cards driving three displays each, we can control nine displays: the eight octagon displays and one operator display. Through massive parallelization over the available CPU cores and graphics cards we are able to achieve real-time visualization even for highly complex 3D scenes [1,7]. The overall system is described in [8] and schematically depicted in Figure 3.

Figure 3: Schematic drawing of the OctaVis system.

Since our VR system provides a perfectly controllable test environment, it allows for custom-tailored neuropsychological diagnosis and training programs. Our system has been evaluated in a multi-centered clinical study with our four medical partners: the clinics for psychiatry, neurology, and epilepsy in Bethel and the Marcus-Klinik for neuro-rehabilitation in Bad Driburg. During a training session the system monitors the patient’s path through the VR environment, logs successful task completions as well as errors, and measures bio-data such as heart rate and skin conductance. This massive amount of data provides insight into the patients’ cognitive deficits as well as their respective training improvements, and will thereby allow us to further improve the training program. Our goal is to prove that the application of real-life-like VR environments can in particular help to improve cognitive functions which are indispensable for the patients’ everyday lives. First results of the clinical studies are very promising [9,12]. In 2013, the CITmed project was awarded with the 3rd place of the Eurographics Medical Prize.

Video: The OctaVis system in action.
OctaVis: A Simple and Efficient Multi-View Rendering System
Eugen Dyck, Holger Schmidt, Mario Botsch
Proceedings of GI VR/AR Workshop, 2010, pp. 1-8.
Virtuelle Realität in der Neurorehabilitation: Evaluation eines 360° VR Supermarkts für das Training kognitiver Leistungen in einem virtuellen Supermarkt
Yasemin Coskun, Inga Hohnemann, Eugen Dyck, Martina Piefke
Jahrestagung der Deutschen Gesellschaft für Klinische Neurophysiologie und Funktionelle Bildgebung (DGKN), 2011
Neuropsychological training of memory functions in a 360°-Virtual reality supermarket and correlations with real-life behavior
Philip Grewe, David Flentge, Agnes Kohsik, Ines Degenhardt, Jan Byszewski, Eugen Dyck, Mario Botsch, Martina Piefke
Jahreskongress der Deutschen Gesellschaft für Neurologie (DGN), 2011
Evaluation of a 360°-Virtual Reality Supermarket for Neuropsychological Assessment and Training of Real-Life Memory Functions
Philip Grewe, Agnes Kohsik, David Flentge, Ines Degenhardt, Eugen Dyck, Mario Botsch, Martina Piefke
Jahrestagung der Gesellschaft für Neuropsychologie (GNP), 2011
Einkaufstraining in einem virtuellen Supermarkt: zwei Fallstudien mit aphasischen Patienten
Katharina Platner, Jennifer Müller, Thomas Brand, Judith Bartoschek, Eugen Dyck, Mario Botsch, Martina Piefke
Jahrestagung der Gesellschaft für Neuropsychologie (GNP), 2012
The role of visual-spatial working memory in map learning and way-finding
Christian Poth, Philip Grewe, David Flentge, Eugen Dyck, Mario Botsch, Martina Piefke
Jahreskongress der Deutschen Gesellschaft für Psychologie (DGPs), 2012
OctaVis: Optimization Techniques for Multi-GPU Multi-View Rendering
Eugen Dyck, Holger Schmidt, Martina Piefke, Mario Botsch
Journal of Virtual Reality and Broadcasting 9(6), 2012
OctaVis: An Easy-to-Use VR-System for Clinical Studies
Eugen Dyck, Eduard Zell, Agnes Kohsik, Philip Grewe, York Winter, Martina Piefke, Mario Botsch
Proceedings of Virtual Reality Interaction and Physical Simulation (VRIPHYS), 2012, pp. 127-136
[PDF]  [Video]  
Learning real-life cognitive abilities in a novel 360° virtual reality supermarket: a neuropsychological study of healthy participants and patients with epilepsy
Philip Grewe, Agnes Kohsik, David Flentge, Eugen Dyck, Christian Bien, York Winter, Mario Botsch, Hans J. Markowitsch, Martina Piefke
Journal of NeuroEngineering and Rehabilitation 10(42), 2013
OctaVis: A Virtual Reality System for Clinical Studies and Rehabilitation
Eduard Zell, Eugen Dyck, Agnes Kohsik, Philip Grewe, David Flentge, York Winter, Martina Piefke, Mario Botsch
Proceedings of Eurographics Short and Medical Prize Papers, 2013 (EG Medical Prize, 3rd place)
Evaluation of Surround-View and Self-Rotation in the OctaVis VR-System
Eugen Dyck, Thies Pfeiffer, Mario Botsch
Proceedings of Joint Virtual Reality Conference, 2013
Real-life memory and spatial navigation in patients with focal epilepsy: Ecological validity of a virtual-reality supermarket task
Philipp Grewe, Denise Lahr, Agnes Kohsik, Eugen Dyck, Hans Markowitsch, Christian Bien, Mario Botsch, Martina Piefke
Epilepsy & Behavior 31, 2014, pp. 57-66
Training in a comprehensive everyday-like virtual reality environment compared to computerized cognitive training for patients with depression
Lorenz B. Dehn, Leona Kater, Martina Piefke, Mario Botsch, Martin Driessen, Thomas Beblo
Computers in Human Behavior 79, 2018, pp. 40-52.
Cognitive training in an everyday-like virtual reality enhances visual-spatial memory capacities in stroke survivors with visual field defects
Lorenz Dehn, Martina Piefke, Max Toepper, Agnes Kohsik, Andreas Rogalewski, Eugen Dyck, Mario Botsch, Wolf-Rüdiger Schäbitz

Involved People

Project Lead
Prof. Dr. Mario Botsch (technical part)
Prof. Dr. Martina Piefke (psychological part)
Ph.D. Students
Dipl. Inform. Eugen Dyck
Dipl. Biol. David Flentge
Dipl. Psych. Philip Grewe
Dipl.-Psych. Agnes Kohsik
M.Sc. Hanno Ohmann
Dipl. Biol. Katharina Platner
M.Sc. Eduard Zell
Student Researchers
Yasemin Coskun
Anna Helwich
Inga Hohnemann
Christian Poth
Central Lab Facilities
Dipl. Inform. Bernhard Brüning
Dipl. Inform. Holger Dierker
Scientific Workshop
Pawel Müller
Clinical Partners
Prof. Dr. Martin Driessen, Clinic for Psychiatry & Psychotherapy, Evangelisches Krankenhaus Bielefeld
Prof. Dr. Wolf-Rüdiger Schäbitz, Clinic for Neurology, Evangelisches Krankenhaus Bielefeld
Prof. Dr. med. Christian G. Bien , Epilepsy Centre, Evangelisches Krankenhaus Bielefeld
Dr. Thomas Brand, Neurorology, Marcus-Klinik Bad Dribung