1. Introduction

The rise of digital communication platforms has transformed how information is created, shared, and consumed. Social media now serves as a key channel for news, providing users with immediate access to global events. However, the absence of strict verification processes and the rapid spread of information have led to the widespread distribution of misinformation and manipulated narratives. This can distort public perception, destabilize social discourse, and pose systemic risks online

[1][2][3][4]

This research develops and validates an automated framework, "FakeKapcha", for detecting misinformation in Russian-language social networks like VK. Instead of relying on manual verification or binary filters, it utilizes a scalable, multi-tiered probability assessment pipeline that addresses localized language and inauthentic behavior [9]. The system uses automated data collection from the VK API, compiling over 350,000 records, then text cleaning and employing high-dimensional TF-IDF feature extraction, the data undergoes processing through a stacked ensemble of eleven machine learning and deep learning algorithms. By combining linguistic signals with metadata, the framework computes a misinformation probability score and classifies content into three categories: Reliable, Potential Misinformation, or Highly Probable Fake Content presented via a user-friendly desktop GUI for real-time analysis [8], [10]. This work presents significant advancements in automated text classification through several interconnected contributions. It introduces a newly curated dataset of over 350,590 publications to address gaps in open-source regional language tools. Our heterogeneous ensemble architecture combines 11 diverse machine learning and deep learning models, establishing a robust predictive baseline for high-dimensional text. This framework offers unique non-binary interpretability, effectively capturing epistemic uncertainty in live data. The findings provide insights into how large-margin linear classifiers and well-regularized probabilistic models manage semantic noise without relying on resource-intensive transformer models. Additionally, this framework is a desktop application for non-technical users like fact-checkers and policy analysts, aiding their efforts against disinformation. This paper details the workflow, pipeline mechanics, classifier performance, current system limitations, and future directions, including user behavior signals and adaptive model retraining.

2. Research methods and principles

The proposed FakeKapcha system detects fake news from social network communities (e.g., VK) using machine learning and natural language processing. It functions as a sequential pipeline, converting raw social media content into structured classification results. Figure 1 illustrates the system architecture, detailing the flow from data collection to output generation.

Data acquisition: textual data and structured metadata, such as content, publication time, and engagement metrics (likes, comments, reposts, views), are fetched from VK communities through authenticated requests to the wall via the VK API. All records are stored in a local SQLite database for processing.

Dataset construction and labelling: the training dataset was created from VK social network data via the official VK API, focusing on verified communities and unverified, low-credibility channels. Low-credibility communities were identified based on the lack of official verification, frequent sharing of unverified content, and manual reputation assessments. Communities focused on entertainment or advertisements were excluded. Publications were labeled: Label 0 for reliable content from verified sources and Label 1 for potentially misleading posts from unverified sources. The final dataset included 350,590 text samples, with duplicates, empty posts, ads, and multimedia-only entries removed to ensure quality.

Experimental Design and Data Partitioning: the dataset, comprising 350,590 labeled VK publications, was divided into training, validation, and test subsets using stratified sampling to preserve class distributions. Specifically, 70% was used for training, 15% for validation, and 15% for testing. This approach ensured balanced representation of both reliable and misleading classes, minimizing sampling bias.

All preprocessing, including text normalization and TF-IDF feature extraction, was performed only on the training set to avoid information leakage. Machine learning models were trained on the same partitions for fair comparisons, and performance was evaluated using Accuracy, Precision, Recall, and F1-score on the unseen test dataset.

Dataset balancing: to address class imbalance, a balancing procedure is applied before model training to ensure that neither class dominates the learning process. This minimizes bias toward the majority class, enhancing model stability and classification reliability.

Text preprocessing: raw text is preprocessed to reduce noise and ensure consistency. This involves converting text to lowercase, removing URLs, punctuation, special characters, and numbers, and then tokenizing, filtering stop words, and normalizing whitespace.

Feature extraction: the cleaned text is vectorized using TF-IDF, a method that emphasizes informative terms while reducing the weight of frequently occurring words to create suitable input for the machine learning models.

Machine learning classification: the prediction engine integrates eleven diverse machine learning classifiers, including Logistic Regression, Support Vector Machines, Random Forest, Extra Trees, Naive Bayes, XGBoost, LightGBM, CatBoost, and a Convolutional Neural Network. Each model generates an independent probability score indicating whether a publication is fake, and the final decision is made by averaging these scores:

where N equals 11 [11]. The ensemble approach enhances stability and reduces overfitting risks compared to individual models (Fig. 3). By aggregating diverse algorithmic predictions, this stacked ensemble improves classification robustness and generalizability, this minimizes model biases and maintains a balanced diagnostic performance [12].

Hardware and Software Specifications:

Training Time and Resource Consumption:

Result generation and visualization: the system outputs the predicted class, probability score, confidence level, original text, and metadata. According to Kulkarni et al. (2021), the use of a three-level output scheme: reliable information, potential misinformation, and fake information, enhances interpretability for uncertain cases [13]. All results are stored in a local SQLite database and presented through a graphical user interface for real-time analysis and visualization.

3. Results and Discussion

The FakeKapcha system was validated using a dataset of 350,590 publications from VK communities via the VK API. Eleven machine learning algorithms were compared under identical conditions using TF-IDF features, with performance assessed by Accuracy, Precision, Recall, and F1-score. The Passive Aggressive algorithm achieved the highest score (0,972), followed by LinearSVC and LightGBM (0,971), XGBoost (0,963), and CNN (0,961). Logistic Regression and CatBoost were around 0,955, while Extra Trees and Random Forest scored between 0,922 and 0,928. Naive Bayes had the lowest performance at 0,896. These results highlight the effectiveness of ensemble learning in creating a strong classification framework that outperforms individual models, as shown by the comparative heat map in Figure 4 [14], [15].

The warmer tones in the heat map indicate high performance for models like Passive Aggressive, LinearSVC, LightGBM, and XGBoost, scoring between 0,955 and 0,972. In contrast, cooler tones for Naive Bayes, Extra Trees, and Random Forest reflect lower performance, ranging from 0,896 to 0,928. This visualization facilitates comparative evaluation of model performance by highlighting outliers and performance gaps without numerical comparisons [16]. The minimal variation across evaluation metrics within each algorithm suggests that top models strike a balance between Precision and Recall, rather than excelling in just one area. However, the uniformity across the heat map confirms that the ensemble improves accuracy while maintaining consistency across all evaluation dimensions, showing that selected base-learners using TF-IDF can effectively classify misinformation without complex architectures [17].

The radar chart (Fig. 5) displays the distribution of Accuracy, Precision, Recall, and F1-score for the top three classifiers: Passive Aggressive, LinearSVC, and LightGBM. Passive Passive Aggressive demonstrates the highest overall performance across all evaluation metrics, indicating superior classification quality, while LinearSVC and LightGBM have slightly lower values. This comparison illustrates that all models maintain balanced performance across metrics, essential for reliable classification and preventing bias. These results align with literature indicating that ensemble architectures, especially with robust TF-IDF representations, enhance discriminatory power compared to individual classifiers [12], [14].

Precision–Recall curves illustrate classification stability across varying decision thresholds. Passive Aggressive and Logistic Regression achieve high precision, with an Average Precision (AP) of 0,989, while Naive Bayes shows a decline in precision as recall increases, resulting in a lower AP of 0,917. This sensitivity to class imbalance is evident in the curves, where Naive Bayes exhibits a more pronounced decline in precision as recall increases before reaching high recall, while Passive Aggressive and Logistic Regression maintain nearly horizontal paths near unity. This demonstrates their ability to preserve high precision even when lowering thresholds to capture more positive instances, contrasting with Naive Bayes’ performance, influenced by feature independence and skewed posterior estimates [18], [19], [20].

PR curves provide a more diagnostic view than ROC plots for the imbalanced VK corpus, as evidenced by the significant AP difference of around seven percentage points between the upper curves and the Naive Bayes trajectory [19]. Passive Aggressive and Logistic Regression demonstrate robustness due to their mathematical foundations: Passive Aggressive effectively adapts to high-dimensional TF-IDF vectors, while Logistic Regression models stable, calibrated probabilities across thresholds. In contrast, Naive Bayes struggles with rare co-occurring terms [18]. Overall, the PR visualization supports the view that large-margin linear classifiers and well-regularized probabilistic models maintain consistent classification quality, while those relying on strong independence assumptions perform poorly as recall increases, highlighting the need for residual-based and ensemble architectures in the FakeKapcha pipeline [21].

The developed application’s operation is shown after completing publication analysis (see Fig. 7). The graphical interface displays 20 analyzed VK posts, including 16 reliable and 4 suspicious publications identified by the classification system. It also includes confidence scores and voting statistics from the ensemble of 11 machine learning algorithms. All evaluation metrics were computed on an independent testing dataset, confirming high classification performance and stable operation of the software when processing large volumes of social network publications.

Practical Utility and Cost-Effectiveness

With the increasing costs of server-grade hardware and limited access to paid cloud AI APIs, the proposed tool offers significant practical benefits for everyday use. Operating as an autonomous desktop application, it allows for real-time verification of VK social media streams without the need for cloud infrastructure, monthly API subscriptions, or expensive GPU hardware. This system serves as a lightweight, privacy-preserving decision-support tool for editors, community moderators, and general users who want to verify local content on VK.

Ethical consideration

Even though automated classification tools offer valuable support, they should not be misconstrued as instruments for determining absolute truth [22]. Machine learning models rely on statistical patterns and may produce errors when handling satire, irony, opinion-based articles, rapidly changing news, or content that lacks sufficient context. As a result, these systems function as decision-support tools that assist users rather than supplanting human judgment. This distinction is vital due to the risks of false positives, which can erode trust in legitimate sources, and false negatives, which may allow misleading content to persist. Transparency remains an ethical imperative; users need to understand that the results are probabilistic estimates, not definitive verifications. Thus, incorporating confidence indicators, probability scores, and a "Potential Misinformation" category helps convey the uncertainty that accompanies automated predictions [23].

Future Directions

Further research and development of the FakeKapcha tool should assertively expand beyond VK, creating a unified platform capable of processing content from other platforms such as Telegram, X, Facebook, Reddit, and other online news portals [4], [13], [22]. is expected to improve detection accuracy.

The tools should be allowed to tap into and expand a dataset and self-model retrain, which will ensure that the system remains resilient and adaptive to evolving information ecosystems.

To maintain long-term reliability, future versions of the system will incorporate continuous performance monitoring using data drift indicators and predictive quality metrics. The ensemble performance will be periodically evaluated through Accuracy, Precision, Recall, and F1-score on newly labeled validation samples. A decrease in the F1-score of more than 5% relative to the baseline performance or statistically significant changes in feature distributions will be used as indicators for initiating ensemble retraining. The retraining process can be planned after accumulating sufficient newly verified data from supported platforms, ensuring that the framework remains robust against evolving misinformation patterns.

4. Conclusion

Modern information environments increasingly require automated verification technologies. This article demonstrates the successful application of machine learning to detect misinformation in Russian-language social media content. The FakeKapcha system implements a comprehensive pipeline, including automated data collection via the VK API, text preprocessing, TF-IDF feature extraction, ensemble classification, and result visualization. Using a custom dataset of over 350,000 publications, we confirmed that this ensemble strategy enhances prediction stability and robustness for heterogeneous textual data. A key innovation is a three-level classification mechanism that categorizes content as Reliable Information, Potential Misinformation, or Fake Information, improving interpretability by enabling nuanced, non-binary assessments alongside transparent confidence scores. The system effectively integrates automated data acquisition, natural language processing, and user-oriented design to assess information reliability on social media. Future development aims to expand platform support, create cross-platform mobile applications, and incorporate multimodal analysis and advanced language models to broaden the system's applicability.

The additional file for this article can be found as follows: