3d point-of-intention determination using a multimodal fusion of hand pointing and eye gaze for a 3d display
Journal article
Authors/Editors
Strategic Research Themes
Publication Details
Author list: Yeamkuan S., Chamnongthai K.
Publisher: MDPI
Publication year: 2021
Volume number: 21
Issue number: 4
Start page: 1
End page: 31
Number of pages: 31
ISSN: 1424-8220
eISSN: 1424-8220
Languages: English-Great Britain (EN-GB)
View in Web of Science | View on publisher site | View citing articles in Web of Science
Abstract
This paper proposes a three-dimensional (3D) point-of-intention (POI) determination method using multimodal fusion between hand pointing and eye gaze for a 3D virtual display. In the method, the finger joint forms of the pointing hand sensed by a Leap Motion sensor are first detected as pointing intention candidates. Subsequently, differences with neighboring frames, which should be during hand pointing period, are checked by AND logic with the hand-pointing intention candidates. A crossing point between the eye gaze and hand pointing lines is finally decided by the closest distance concept. In order to evaluate the performance of the proposed method, experiments with ten participants, in which they looked at and pointed at nine test points for approximately five second each, were performed. The experimental results show the proposed method measures 3D POIs at 75 cm, 85 cm, and 95 cm with average distance errors of 4.67%, 5.38%, and 5.71%, respectively. © 2021 by the authors. Licensee MDPI, Basel, Switzerland.
Keywords
Hand recognition, Hand tracking, Multimodal systems
Contact Information