Author: Matjaž Divjak
Mentor: Prof. Dr.  Damjan Zazula
Co-Mentor:Prof. Dr. Nikola Guid
Date: March, 2003

3D MOTION TRACKING USING A STEREOCAMERA

Keywords: motion tracking, stereocamera, tracking devices, computer vision, digital image processing, stereoscopic image, depth image, Kalman filter, prediction-correction methods

UDK: 004.354: 004.92

Abstract: Several approaches for tracking the movement of objects in 3D space exist. Most of them actually track the motion of special sensors attached to the object of interest. This method is often less appropriate, because it obstructs the free movement of the object. However, some approaches depend only on natural properties of objects and don’t require any special hardware.

In this work we present the design, implementation and characteristics of the tracking system which uses stereoscopic camera to detect the motion. We focus on tracking the face and hand movements of the person in front of the camera. First, the depth image is computed from the two stereo images and then the input data is segmented in two ways: regarding the skin colour and regarding the depth. Partial results are merged together and only the most promising ones are kept. Next we determine the correspondence between the currently detected objects and the objects from the previous image and compute their geometric centres of gravity. Coordinates of the centres of gravity are linked together to form the movement trajectories. With this procedure we find 97 % of objects of interest and correctly detect up to 70 % of their surface. The tracking method is then upgraded with the prediction-correction algorithm. As a result, smoother trajectories and object borders are obtained. The trajectories are compared with the results of the magnetic motion tracking device which serves as a reference. The average difference between the calculated and reference trajectories yields approximately 1.75 cm.

Next we present the main factors which contribute to the error of the method and estimate the time complexity of the used algorithms. Face and hand tracking is extended to tracking of arbitrary objects, several examples of possible applications are discussed, the main drawbacks are exposed and their solutions are presented. Despite the prototype implementation our method achieved satisfactory results. We therefore believe it provides a good basis for further research.