Semester Topic: Analysis and Synthesis of Images
Group Members: Carsten Høilund, Jeppe Jensen, and Simon J. K. Pedersen
Abstract:
The objective of this project is to develop a real-time augmented reality system with correct occlusion. In order to create correct occlusion depth information of the real scene is needed. This depth information is acquired with the Bumblebee XB3 stereo vision system as disparities between two images, a so-called disparity map. Due to the quality of these disparity maps they can not immediately be used for correct occlusion.

From an analysis of the disparity maps five main issues are detected: Occlusion shadows, coarse edges, holes, wrong disparity values, and frame rate.

To solve these issues the main method applied is background subtraction (the codebook algorithm). The information from the background subtraction algorithm about foreground and background objects is combined with the disparity map to create an improved disparity map. Along with background subtraction other algorithms are used to improve the disparity map: Disparity trimming, median filtering, extrapolation, background disparity modeling, and region of interest. All these algorithms are merged into one coherent system.

The test of the system showed that the values of the disparity map were improved slightly and quality of the edges was improved greatly in addition to a significant increase in frame rate.

Not all problems in regards to correct occlusion have been solved in this project, but the combination of background subtraction and the other proposed solutions to repair the disparity maps have brought correct occlusion one step closer.