Trajectory tracking of a robot arm using images sequences

Abstract

In this project, which belongs to the field of computer vision and robotics, we use the SCARA two -degree- of- freedom robotic arm to generate trajectories through a combination of two methods. The first part of our method is the CNN, which is used to detect and locate the object concerned by tracking a sequence of images acquired by the arm during its movement in a 2D plane. Spline3 is used to generate a trajectory using the polar coordinates found by the CNNs. because it generates a set of given points while minimizing unwanted oscillations and irregularities. Simulations are acquired with a model of two degree of freedom (2-d.o.f.) of the manipulative arm SCARA has proven that the better the object is located, the better the trajectory tracking precision is. The work carried out is evaluated using metrics: the OD part uses mAP, recall, and precision; the creation of the trajectory is evaluated by cosine similarity, MSE, and PSNR. The primary objective of this work is to support surgeons during surgery by following their path and then using the SCARA arm to create an identical path. As an early effort, we tracked a simple object with a simple trajectory.