Hand Gesture Recognition for Collaborative Workstations: A Smart Command System Prototype

Human-machine collaboration is a key aspect in modern industries, which must be compliant to the Industry 4.0 paradigm. Although the collaboration can be achieved using a Collaborative Robot in a purposely designed workstation, this solution is not always neither feasible nor affordable for the specific task to be carried out in the workstation. On the other hand, using a smart HMI to make an industrial robot a “smart” robot can be a better and affordable solution depending on the task. In this work we present the preliminary development and characteristics of an experimental HMI for smart manufacturing developed in MATLAB and ROS Industrial. The collaboration between humans and robots is achieved by leveraging the Faster R-CNN Object Detector to robustly detect and recognize the hand gestures performed in real-time. The system is based on a state machine to carry out simple tasks such as the repeated movement of the robot following a given trajectory and a pick and place task where the robot interactively reaches a given point and a jog modality.