3D POI Determination using a Multimodal Fusion of Eye Gaze and Hand
Pointing for a 3D Display

This project proposes a 3D point of intention (POI) determination method using

multimodal fusion between eye gaze and hand pointing for a 3D virtual display. In the

method, the finger joint forms of the pointing hand sensed by a Leap Motion sensor are

detected as pointing intention candidates, and differences with neighboring frames, which

should be during hand pointing period, are checked by AND logic with the hand pointing

intention candidates. A crossing point between the eye gaze and hand pointing lines is

finally decided by the closest distance concept. To evaluate the performance of the

proposed method, experiments with some participants may prove effectiveness of the

proposed method.