Human computer interaction for 3D model visualization using sensor fusion

Abstract

In the last years, three dimensional visualization has become a widely extended tendency, including this technology on televisions or even Smartphones. Moreover, 3D imaging has been also introduced on medical diagnosis in the last years. However, volumetric objects have a rough interaction because the devices used to control them are bi-dimensional. For that reason, wearable sensors will give a three-dimensional control for this visualization helping the user to interact in a more natural manner. This kind of control has been a common subject in computer vision research, even more with the introduction of Microsoft R Kinect TM , but in the wearable sensor area, this is not a commonly extended subject of research and there are even less works including sensor fusion. This project proposes a system for managing a volume visualizer including in the development the whole system from the sensor data acquisition to the model visualization. The system will consist on two wearable sensors (with accelerometer and gyroscope) that will be attached to the arms of the user, an android phone will stream the values acquired from the sensors (via Bluetooth R communication) to the Server using a wireless TCP/IP protocol. The server will store the values in the computer and a 3D Engine will process this data using a complementary filter to fuse the acceleration (given by the accelerometer) and the angular velocity (given by the gyroscope) in order to obtain the performed action and apply the required transformations to the displayed model. In order to validate the system, a real-case example will be used, trying to find some anomalies on a 3D textured model of a heart in a simulated medical diagnose process. This project tries to build a complete system of natural interaction between the human and the computer by using wearable sensors.