Authentic Signature Verification Using Deep Learning Embedding With Triplet Loss Optimization And Machine Learning Classification

Abstract

Various document types (financial, commercial, judicial) necessitate signatures for authentication. With the advancements of technology and the increasing number of documents, traditional signature verification methods encounter challenges in facing tasks related to verifying images, such as signature verification. This idea is further reinforced by the growing migration of transactions to digital platforms. To that end, the fields of Machine learning (ML) and Deep Learning (DL) offer promising solutions. This study combines Convolutional Neural Network (CNN) algorithms, such as Visual Geometry Group (VGG) and Residual Network (ResNet) or VGG16 and ResNet-50 specifically, for image embedding alongside ML classifiers such as Support Vector Machine (SVM), Artificial Neural Network (ANN), Random Forest, and Extreme Gradient Boosting (XGBoost). While the aforementioned solutions are usually enough, real life scenarios tend to differ in environment and conditions. This problem leads to difficulty and accidents in the verification process, causing the users to redo the process or even end it prematurely. To alleviate the issue, this study employs optimization methods such as hyperparameter tuning via Grid Search and triplet loss optimization to enhance model performance. By leveraging the strengths of CNNs, Machine Learning classifiers, and optimization techniques, this research aims to improve the accuracy and efficiency of signature verification processes while addressing real-world challenges and ensuring the trustworthiness of electronic transactions and legal documents. Evaluation is conducted using the ICDAR-2011 and BHSig-260 datasets. Results indicate that triplet loss optimization significantly improves the performance of the VGG16 embedding model for SVM classification, notably elevating the Area Under the ROC Curve (AUC) from 0.970 to 0.991.