Обнаружение фальсифицированных подписей с применением глубинного обучения

Biometrics use physiological or behavioral attributes to verify an individual’s identity. Unlike passwords or keys, biometric traits are unique and cannot be lost, making them a reliable means of authentication. Among behavioral attributes, handwritten signatures remain one of the most widely used means of authentication in financial institutions, insurance, monetary transactions, etc. Therefore, signature verification, i.e., determining whether a signature is genuine, remains a critical task. However, manual verification is tedious and time-consuming, which has led to the development of automated signature verification systems based on machine learning

Signature verification systems are generally divided into online and offline approaches. Online systems capture dynamic features such as pen pressure, which makes forgery detection easier but requires special input devices and is therefore less common in practice. This study focuses on offline signature verification (OSV), where signatures are represented as images obtained from scanners, making it more difficult to detect a fake. Among the three main types of offline forgeries — random, simple, and skilled — this research focuses on skilled forgeries as the most challenging ones

OSV remains challenging due to high intra-personal and the resulting low inter-personal signature variations. Existing OSV methods commonly rely on convolutional neural networks (CNNs), which effectively extract multi-level local features through convolutional and pooling layers

Thus, the objective of this study is to develop and evaluate a transfer learning-based approach for the WI OSV task using modern hybrid computer vision architectures, aimed at improving the detection of skilled forgeries under limited-data conditions.

The novel contributions of this work are as follows:

1. Hybrid ConvNeXt

2. The proposed architectures are evaluated under two training schemes: standalone classification and Siamese networks to assess their applicability to OSV from both direct classification and similarity-learning perspectives. Several verification strategies are compared, including Euclidean distance-based verification and cosine similarity-based verification.

Generally, OSV methods consist of three stages: data pre-processing, feature extraction, and signature verification.

2.1. Data preprocessing

Building OSV systems has proven to be a challenge since data acquisition is done under different conditions. To address variability in background artifacts and remove irrelevant patterns, several preprocessing steps are performed. In the initial phase, the Otsu threshold algorithm

Sample signature:

a) before preprocessing; b) after preprocessing

DOI:10.60797/IRJ.2026.172.87.1

display_fulldownload_image

2.2. Feature extraction

For feature extraction, ConvNeXt and CoAtNet are pretrained on ImageNet-21K

2.2.1. ConvNeXt

ConvNeXt is a model made up of CNN modules that incorporates various Transformer-inspired techniques

The model outperforms several state-of-the-art vision architectures, including Swin Transformer, DeiT, and ViT, while remaining competitive with EfficientNet and EfficientNetV2 and preserving the simplicity of CNNs. Since ViT has already shown promising results in OSV, ConvNeXt may also be effective for this task. Its successful adaptation to related visual recognition problems, such as emotion recognition

2.2.2. CoAtNet

CoAtNet is based on integrating convolution and self-attention within computational blocks and stacking them into a unified network

CoAtNet outperforms ViT, Swin, and EfficientNet on ImageNet-1K, ImageNet-21K, and JFT-300M, demonstrating better scaling efficiency than pure CNNs and Transformers. These results motivate its application to OSV.

2.3. Signature verification

During this stage, the query signature is compared with the reference in the embedding space. In forensic tasks, it is reasonable to look at the spatial relationships between the query and the original embeddings when identifying the likeness of the former to the latter. Thus, after training, the classifier head is removed, and each model is used solely for embedding extraction. Two distance metrics are evaluated to measure signature similarity.

2.3.1. Euclidean distance

One of the most frequent techniques of verification

Accepting a pair of embeddings (reference, query), Euclidean distance is calculated between two signatures, following the method described in

At first, m random genuine signatures from each user are selected. The distances between them and the reference signatures are calculated. The results are aggregated into one array of numbers with maximum and minimum defined as:

where n is m × (# signatures used per person). A set of candidate thresholds is then generated with a fixed step:

The optimal threshold d from D is selected on the training set, with mm varied from 1 to 12 and the step size fixed at 0.01.

2.3.2. Cosine similarity

Cosine similarity measures the angle between the embeddings vector. Therefore, it is less sensitive to pen pressure, resolution, and brightness, focusing instead on shape-related differences between embeddings. During experiments, m genuine signature embeddings are averaged into one. This improves robustness by preserving the main biological pattern while averaging out random variations. A query is classified as genuine if its cosine similarity exceeds the threshold.

Experiments are conducted over 3 OSV benchmark datasets: GPDS Synthetic

Example of signature samples from different datasets:

a) UTSig; b) CEDAR; c) GPDS Synthetic

DOI:10.60797/IRJ.2026.172.87.5

display_fulldownload_image

Models are trained only on GPDS Synthetic, while CEDAR and UTSig are used for external evaluation. Data is split strictly by users to prevent overfitting to user-specific features. GPDS users No.301–1000 are split into 90% training and 10% validation sets, while users No.1–300 form the test set. All CEDAR and UTSig users are used for evaluation.

Two training approaches are evaluated. First, standalone models are trained with Cross-Entropy Loss to classify individual signatures and assess their ability to extract discriminative and relevant features from individual images.

Since real-world verification usually involves a reference signature, in the second approach the discussed models are used as Siamese backbones. This improves generalization by focusing on relative feature relationships rather than class labels. Reference and query signatures are processed with shared weights and trained using Triplet Loss. For each user, triplets consist of an anchor, a randomly selected genuine positive, and a forged negative. Each genuine signature is used once as an anchor, producing 15,120 training triplets.

All models are trained on an NVIDIA Tesla V100 using Adam optimizer and a batch size of 32 over 50 epochs with a constant learning rate of 1e-5.

The following metrics are considered in evaluating the performance of verification systems.

1. Accuracy denotes the ratio of correctly identified signatures to all the available test data.

2. EER displays the error rate at which FAR (False acceptance rate) equals FRR (False rejection rate).

Both of these metrics are said to be the most informative in the signature verification field

Results are shown only for best-performing model sizes based on accuracy.

Compared with state-of-the-art models, CoAtNet and ConvNeXt generally achieve lower performance on CEDAR and GPDS Synthetic datasets (Table 4, Table 5 respectively).

However, the Siamese ConvNeXt-B and CoAtNet-B models reach accuracies of 80.0% and 78.9%, respectively, outperforming SigNet on GPDS Synthetic. Despite its rare use in OSV, cosine similarity yields better results for most models than the more frequently used Euclidean distance on GPDS Synthetic. The reason for this may lie in the specifics of the database. Contrary to the others used in this work, GPDS Synthetic dataset contains images with high variations in pen pressure as demonstrated in Figure 2. Therefore, as cosine similarity is less sensitive to such features, it produces greater performance for the discussed models.

However, when tested on UTSig dataset (Table 6), models outperform other architectures with Siamese ConvNeXt-B achieving the best accuracy of 82.3% and EER of 16.6.

A possible explanation lies in the specifics of the datasets. As shown in Figure 2, in general UTSig dataset contains signatures with more complex elements than the smooth and sometimes fragmented curves of GPDS Synthetic due to the characteristics of the Persian script. Therefore, due to architectural details (self-attention in CoAtNet and hierarchical feature extraction in ConvNeXt) the proposed models adapt more easily to diverse Persian glyphs, whereas on simpler instances of Western languages they are prone to overfit to irrelevant details. The reason for underperformance on the CEDAR dataset might be the low resolution of images in this dataset, to which the models are not adapted yet, although further experiments are needed to confirm this.

It is noticeable that Siamese networks perform the best on all of the used datasets. Both ConvNeXt and CoAtNet have been designed for image classification, which does not require attention to such intricate details as signature verification does. Since standalone model training is based on models being capable of extracting only features relevant to the task, the chosen models underperform, overfitting to superficial patterns. However, using Siamese networks and Triplet loss, one can refocus these models for a similarity learning task instead of classification

This work presents a transfer learning-based approach to Writer Independent offline signature verification using the modern ConvNeXt and CoAtNet architectures. Their performance is observed under standalone classification and Siamese training schemes. Euclidean distance and cosine similarity are used for embedding-based verification.

The main findings are as follows:

1. ConvNeXt and CoAtNet models achieve their best results on UTSig with 82.3% and 81.76% accuracy with Siamese training, but mainly underperform the other state-of-the-art models on the remaining datasets with the exception of SigNet.

2. The applicability of the proposed models depends on signature complexity and image quality: they perform better on structurally complex Persian signatures but may overfit simpler Western-style or low-resolution samples.

3. The choice of verification metric for such models should reflect the variability of the data. Cosine similarity achieves better results on GPDS Synthetic since it is generally more robust than Euclidean distance to variations in pressure, brightness, resolution, and scanning conditions.

The scientific significance of this work lies in defining the applicability and limitations of ConvNeXt and CoAtNet in OSV and examining how their performance depends on the training scheme, verification metric, and dataset properties.

Future research may focus on simplifying these architectures to avoid overfitting on signatures with lesser details.