Trajectory tracking of a robot arm using images sequences

Abstract

In this research paper, in order to build trajectories, we used a robotic arm in computer vision and robotics. The combination of two methods, the first of which is used in this work, is a deep learning method based on convolutional neural networks (CNN); the second is Spline3, which is utilized to achieve accurate trajectory tracking. The CNN is employed to track a sequence of images acquired by the robotic arm while moving in a 2D plane. The CNN correctly locates and identifies the object by examining the visual data. Once the object is detected and located by the CNN, the outcome information is saved in a txt file. The next step is to generate a trajectory using the Spline3 method and the created txt file. This trajectory has the property of minimizing oscillations and irregularities, which ensures accurate path generation. Simulations are performed using a two-degree-of-freedom model of the SCARA arm in order to assess the efficacy of the suggested technique. These simulations demonstrate the relationship between accurate object localization by the CNN and trajectory tracking precision by the robotic arm. The metrics used for the evaluation of the proposed method include mean average precision (mAP), recall, precision, cosine similarity, mean squared error (MSE), and peak signal-to-noise ratio (PSNR). The metrics provide quantitative values of object detection accuracy by CNN and path generation similarity by Spline3. The main aim of this study is to enable the use of this type of manipulator arm in the most complex areas, for example, to help surgeons carry out their surgical operations in an accurate and reliable manner.