Laboratory for Image and Media Understanding,Department of Advanced Information Technology, Kyushu University

Early Understanding of Gesture Patterns based on Behavior

Outline

We aim at establishment of early gesture recognition which copes with “Quickness of Service” and “Quality of Service”. We research technique of “earlier” and “more correct” recognition which reflects “habit” and “hesitation” of users by the combination of the third person sensing by the sensor in fixed surroundings and the first person sensing by the sensor wore to users.

Detail

Man-machine interaction which uses a gesture of users becomes popular, Wii (Nintendo) etc. In these days, the measuring instruments which can obtain the posture data timely become practical, for example, Kinect sensor. Therefore, the interfaces which use human gestures for input are desired to realize. Human gestures are variable information by time and space and that information has the unique habit and hesitation of users (we call “behavior patterns”). So it is very difficult to recognize the information which users want to inform of systems (intention of gestures) immediately and correctly like controlling with button. Therefore, the systems are required the compatibleness; the guarantee of “Quality of Service” which is high quality output that considered the behavior patterns, and the “Quickness of Service” which is quick interpreting of users’ intention on the gestures.

In the previous research, we focused the “Quickness of Service”. We proposed “early recognition” method which output the recognition result before users’ gesture has not completely finishes by obtaining the feature of users’ bodies from the camera in fixed surroundings. On the other hand, about “Quality of Service”, we did the recognition by the recognition system which is common to all users because we did not have the recognition system which can understand the behavior patterns in each user. Therefore, users have to adjust their own gestures to the system, and that decreases the “Quality of Service”.

We aim at establishing the way of early recognition which supports “Quickness of Service” and “Quality of Service” at the same time. The system needs to understand the intention in the users’ behavior patterns to realize the interaction that users can feel high “Quality of Service”. In previous research, the way of estimating intention from features of users’ gestures by the third person sensing which uses the camera in fixed surrounding. However, this way is difficult to obtain the users’ behavior patterns because this way cannot capture the features of fine parts of bodies from the problem of resolution and processing speed. Then, it was very difficult to obtain the behavior patterns only from objective viewpoint like the third person sensing because behavior patterns exist in subjective gesture of users essentially. On the other hand, Introduction of the technique of analyzing self-gestures by the first person sensing which uses the sensors wore to users enables us to obtain the points of gaze of users and self-gestures directly. The first person sensing have advantages that it can obtain the intention of users’ gestures easily because compared with the third person sensing, this sensing obtains the more detail information of gestures. We are researching the technique that obtains the unique habits and behaviors of users in detail and recognizes at earlier timing correctly.

Research Promotion

This work was supported by KAKENHI Grant-in-Aid for Young Scientists (A) (23680018)