Real-Time Human Action Recognition using OpenPose and Sequence-Based Classification

Abstract

Human Action Recognition is one important area of Artificial Intelligence that is still in development. The ability to recognize action in human objects will significantly increase the understanding of images or videos for many practical purposes. This research employs three sequence-based algorithms to detect human actions, which are LSTM, CNN-LSTM, and CONVLSTM, to predict human action sequences in videos. The steps taken are 1) Collect action videos from video clips of actions as the data source. Convert the video clips into data sets for model training and testing. 2) Build the model using the datasets and the selected sequence-based classification algorithms. The best model from each algorithm is then implemented to get the inference engines. 3) Build inference engines for each algorithm. Action videos are collected and extracted by their key points using OpenPose; these 30 frame key points data are used to train the models. The results are the ability to predict seven human actions with an accuracy of 83.1429% in the LSTM model, 83.7143% in the CNN-LSTM model, and 83% in the CONVLSTM model. Inference engines for these models converted in TFLite were built to demonstrate that the systems can detect real-time action in recorded or webcam. The TFLite versions of LSTM, CNN-LSTM, and CONVLSTM inference times are 0.5ms, 0.25ms, and 0.5ms, respectively.