Abstract: People are gradually abandoning the mouse and keyboard since touch screens come into their lives, however the shortcomings and limitations of touch manipulation emerge subsequently, thus non-contact manipulations are becoming the new trend of human-computer interaction in the future due to its unique advantages. There are many problems in existing non-contact manipulation approaches, e.g. improper definitions of gestures, complex algorithms, low recognition accuracy, etc. This paper proposes RemoteControl, a non-contact manipulation approach based on machine learning and the Kinect device. Users can finish most fundamental GUI (graphical user interface) manipulations in the 3-dimensional space by the two actions, “grab” and “put”. RemoteControl utilizes SVM (support vector machine) and random forest to recognize hand shapes in hand images. The experimental results show that RemoteControl achieves high accuracy in hand gesture recognition.

Key words: human-computer interaction, non-contact manipulation, gesture recognition, Kinect, machine learning

摘要： 触摸屏的出现使人们逐渐摒弃鼠标和键盘，而触摸操作的不便和局限性也随之显现，非接触式操作凭借其独特的优势正逐渐成为未来人机交互方式的新潮流。针对现有的非接触式操作方式存在的手势定义不合理，方法复杂难解，识别精度低等许多问题，提出了使用“抓”和“放”这两个手势的非接触式操控方式RemoteControl，用户可在三维空间中做出操控动作来完成大部分基础图形用户界面（graphical user interface，GUI）操作。RemoteControl使用支持向量机（support vector machine，SVM）和随机森林来从手部图像中识别出手的形状。实验结果表明RemoteControl的操控动作识别取得了较高的精确度。

关键词: 人机交互, 非接触式操作, 手势识别, Kinect, 机器学习