Securing Online Job Platforms: A Distributed Framework for Combating Employment Fraud in the Digital Landscape

Moving on to the discussion of the result, it is considered that the dataset includes classes 0 (real job) and 1 (fake job), with a class rate of 95% and 5%, respectively. The initial DataFrame has 17880 records and 18 features. In subsequent experiments on the test data, all classifiers use a stratified split of 85% training data and 15% testing data to maintain a balanced distribution between real and fake job postings. Outliers are removed from the numerical features before conducting the experiments, and the evaluation metrics are presented through column charts that display precision, recall, and F1-score for both classes and overall accuracy.

In addition, this work utilizes both CML and DML models to assess their performance and scores. In the context of CML, various classifiers are employed including Decision Tress (DT), Random Forest (RF), Logistic Regression (LR), Boosting with DT (BDT), Bagging with DT (BagDT), Boosting with Random Forest (BRF), Bagging with Random Forest (BaggRF), Stochastic Gradient Descent (SGD), Nearest Centroid (NC), and Perceptron algorithms [30]. Within the DML several classifiers are employed including Distributed Decision Trees (DDT), Distributed Random Forest (DRF), Distributed Logistic Regression (DLR), Distributed Gradient-Boosted Tree (DGBT), Distributed Linear Support Vector Machine (DLSVM), Distributed Naive Bayes (DNB), and Distributed Factorization Machines (DFM) classifiers are applied [31, 32].

Table 3. Null count and percentage of categorical features

Table 4. Null count and percentage of textual features

Table 5. Null count and percentage of numerical features

Five experiments are conducted for each detection engine, focusing on managing missing values and addressing categorical, textual, and numerical features. In the first experiment, all missing values are included without dropping any feature. In the second experiment, missing values of textual features are dropped while other missing values are included. In the third experiment missing values of categorical and textual features are dropped while numerical missing values are included. The fourth experiment drops all missing values. Finally, the fifth experiment drops features with many missing values and keeps relevant features. Through all experiments, it is assumed that if a textual or categorical feature has missing value, it will be replaced with the term 'missing'. However, any numerical feature with missing values is replaced with the number '0'.

Tables 3-5 illustrate null_count and null_percentages, which are used for categorical, textual, and numerical features in both CML and DML, respectively.

4.1 Using conventional machine learning (CML) classifiers

As mentioned earlier, the employed CML classifiers are Decision Trees (DT), Random Forest (RF), Logistic Regression (LR), Boosting with Decision Trees (BDT), Bagging with Decision Trees (BagDT), Boosting with Random Forest (BRF), Bagging with Random Forest (BagRF), Stochastic Gradient Descent (SGD), Nearest Centroid (NC), and Perceptron algorithms. The subsections describe a detailed account of the performance of various experiments conducted using different CML models.

4.1.1 Dealing with all missing values without dropping any features

This experiment aims to address missing values without dropping any features. The class distribution remains consistent, with Class 0 accounting for 95% and Class 1 for 5%. The resultant DataFrame maintains the same 18 features but has a modified shape, consisting of 4965 records. Figure 4 (a) illustrates scores of CML classifiers using all features. Notably, all classifiers showcase excellent recall, precision, and F1-score for Class 0, resulting in impressive overall accuracy of 99%. Regarding fake job postings, the BDT and BagDT classifiers slightly underperform compared to Class 0 but still maintain a high accuracy level of 99%. On the other hand, the SGD classifier presents comparatively weaker results, achieving a moderate accuracy of 94%. Besides, the NC and Perceptron classifiers display relatively minor results for both classes, especially for Class 1, with moderate accuracy levels (83% for Perceptron and 85% for NC).

4.1.2 Dropping missing values of textual features while handling other missing values

This experiment aims to systematically address the presence of missing data, contributing to a more comprehensive analysis of the dataset by mitigating the impact of missing values on the features' textual and numerical aspects. The class distribution reveals that Class 0 constitutes the majority, accounting for 96%, while Class 1 comprises a smaller proportion at 4%. The resulting DataFrame consists of 2846 records and includes 18 features. Figure 4 (b) shows that all classifiers showcase excellent recall, precision, and F1-score for Class 0. Regarding Class 1, while the DT, RF, LR, Boosting, and Bagging classifiers maintain consistent excellence in Class 1 and achieve a perfect accuracy rate of 100%, the BRF and BagRF classifiers have a slight reduction in recall. Otherwise, the SGD, NC, and Perception classifiers encounter challenges in Class 1. Although the SGD classifier slightly diminishes the F1-score, it maintains a commendable accuracy rate of 96%. On the other hand, the NC classifier achieves an overall accuracy of 81%. Besides, the Perceptron classifier struggles to identify Class 1, resulting in a 0% precision, recall, and F1-score, with a minor effect on the overall accuracy of 96%.

4.1.3 Dropping missing values of textual and categorical features while handling numerical missing values

In this experiment, missing values in categorical and textual features are removed, while numerical feature values are replaced by the numeric value '0'. Analysis of the class distribution exposes that Class 0 constitutes the majority of 94%, while Class 1 comprises a smaller proportion at 6%. The resultant DataFrame consists of 2207 records, and 18 features.

The CML classifier scores are illustrated in Figure 4 (c). This figure shows that all classifiers accurately identify Class 0, achieving flawless recall, precision, and F1-score. Concerning fake job postings (Class 0), the DT, RF, LR, and Boosting classifiers demonstrate excellent performance with high overall accuracy ranging between 99% and 100%. On the other hand, the BagDT, BagRF, NC, and Perceptron classifiers show a relatively lower performance. Whereas the BagDT and BagRF classifiers keep up a remarkable accuracy rate of 99%, the SGD achieves an overall accuracy of 94%, the NC classifier attains an overall accuracy of 73%, and the Perceptron classifier reaches an accuracy rate of 94%.

4.1.4 Dropping all missing values

This experiment handles missing values within all features and aims to create a more refined and balanced dataset for subsequent analysis and modeling. The resulting class distribution indicates 90% for Class 0 and 10% for Class 1. Consequently, the shape of the DataFrame is modified to comprise 720 records and 18 features. As shown in Figure 4 (d) Class 0 achieves complete accuracy across all classifiers. As regards to Class 1, the RF and Boosting classifiers achieve a 100% accuracy rate, while the LR maintains 97% accuracy. However, the NC and SGD face challenges, resulting in lower precision, recall, and F1-score.

4.1.5 Dropping features with many missing values and keeping relevant features

In this experiment, certain features such as job_id, title, location, department, telecommuting, has_company_logo, has_questions, employment_type, required_experience, industry, and function fetures are dropped and only company_profile, description, requirements, benefits, min_salary and max_salary are utilized in the training and testing of the CML models. The resultant class distribution reveals 16% for Class 0 and 84% for Class 1. Consequently, the resultant DataFrame consists of 4965 records and 9 features. As shown in Figure 4 (e) all classifiers demonstrate high recall, precision, and F1-score for Class 0. Concerning Class 1, the DT, RF, LR, Boosting, and Bagging classifiers attain high recall, precision, and F1-score, showcasing strong performance with high accuracy (reaching 99%). Besides, the SGD classifier shows moderate performance with a 96% accuracy rate. Both the NC and the Perceptron classifier classifiers face challenges, leading to a notable decrease in recall, precision, and F1-score, with an 85% accuracy rate.

4.2 Using distributed machine learning (DML) based on Apache Spark framework classifiers

This subsection describes the system performance of the DML classifiers, including Distributed Decision Trees (DDT), Distributed Random Forest (DRF), Distributed Logistic Regression (DLR), Distributed Gradient-Boosted Tree (DGBT), Distributed Linear Support Vector Machine (DLSVM), Distributed Naive Bayes (DNB), and Distributed Factorization Machines (DFM), based on Apache Spark when applying the above mentioned five experiments.

4.2.1 Dealing with all missing values without dropping any features

As mentioned above, this experiment aims to address missing values without dropping any features. Figure 5 (a) presents the scores of the DML classifiers when using all features. Similar to the CML classifiers, all DML classifiers showcase excellent recall, precision, and F1-score for Class 0, resulting in impressive overall accuracy. Regarding fake job identification, the DDT and DRF classifiers face challenges resulting in a lower F1-score due to compromised recall and precision. Besides, the performance of the DLR classifier, especially in recall, is moderate. Moreover, the DGBT and DLSVM classifiers display decent precision; however, their recall is relatively low. In contrast, the DNB classifier has lower overall metrics and accuracy. Lastly, the DFM classifier demonstrates balanced performance for both classes, achieving an accuracy rate of 95%.

4.2.2 Dropping missing values of textual features while handling other missing values

Figure 5 (b) represents the textual features based on DML classifier scores. As per the previous experiment, all classifiers exhibit outstanding performance with the highest F1-score, recall, and precision for Class 0, maintaining a high overall accuracy. Regarding the fake job identifying, the DDT, DRF, DLR, DGBT, and DLSVM classifiers demonstrate almost flawless metrics, with slightly lower performance but still retained a commendable accuracy of 99%, 98%, 98%, 98%, and 98% respectively. On the other hand, both the DNB and DFM classifiers encounter challenges with Class 1. While the DNB classifier has an overall accuracy rate of 82%, the DFM maintains an accuracy rate of 97%.

4.2.3 Dropping missing values of textual and categorical features while handling numerical missing values

Figure 5 (c) illustrates the performance of various DML classifiers. Missing values of textual and categorical features are dropped while any missing numerical feature values are replaced with '0'.

4.2.4 Dropping all missing values

Figure 5 (d) illustrates the performance of various DML classifiers after dropping all missing values of textual, categorical, and numerical features. Like the previous experiments, all classifiers exhibit outstanding performance with the highest F1-score, recall, and precision for Class 0, maintaining a high overall accuracy. However, for Class 1, the DDT and DRF classifiers demonstrate a moderate performance, resulting in an overall accuracy rate of 97%. Besides, the DLR and DGBT classifiers exhibit variable performance. The DLSVM classifier struggles to achieve 91% accuracy, and the DNB classifier has low recall and precision, leading to 55% accuracy. Lastly, the DFM classifier fails to identify Class 1, slightly impacting its overall accuracy of 91%.

4.2.5 Dropping features with many missing values and keeping relevant features

Different DML classifiers are evaluated when missing values are dropped, as shown in Figure 5 (e). All classifiers show impressive performance metrics for Class 0. Regarding Class 1, the performance of the DDT and DRF classifiers is moderate, with lower recall, F1-score, and precision. While the DLR, DGBT, and DLSVM classifiers have varied performance, the DLR classifier has comparatively worse precision, recall, and F1-score, resulting in an accuracy rate of 97%. On the other hand, the DNB classifier comes across challenges, impacting its overall accuracy (rate 92%). Besides, the DFM classifier struggles significantly with Class 1, affecting its overall accuracy at 85%.

11111.png

(a)

42.png

(b)

43.png

(c)

44.png

(d)

45.png

(e)

Figure 4. Conventional machine learning classifier scores

51.png

(a)

52.png

(b)

53.png

(c)

54.png

(d)

55.png

(e)

Figure 5. Distributed machine learning classifier scores

4.3 Assessing CML and DML performance through k-fold cross-validation and accuracy learning curves

There are several evaluation methods that determine the accuracy of machine learning models; in this work, k-fold cross-validation and accuracy learning curves are used [33-37]. Since the dataset is imbalanced, F1-scores are used as a reliable metric for evaluating the best-performing machine learning classifier for both CML and DML.

4.3.1 Examining CML and DML performance using k-fold cross-validation

Cross-validation is a method for evaluating the predictive performance of ML models on unseen data [33-35]. It splits the data into ‘k’ subsets and creates training and validation sets. The purpose of cross-validation is to provide a more reliable and unbiased measure of different classifiers' performance, as compared to a single train-test split. Figure 6 illustrates a comparison of various classifiers applied to the five aforementioned experiments on the CML training data. Regardless of the experiment, the SGD, NC, and Perceptron classifiers consistently exhibit the lowest F1-scores. In Experiment #1, the LR classifier achieves the highest F1 scores, and the RF classifier ranks second. Additionally, the DT classifier takes third place with good F1-scores. However, the boosting and bagging for both the RF and the DT do not contribute to improvement. In Experiment #2, the DT classifier obtained the highest F1-scores, while the boosting and bagging could not enhance the scores. The RF classifier ranks second, but its boosting and bagging techniques do not improve the F1-scores. The LR classifier comes in third place. Experiment #3 shows that the RF classifier performs the highest F1-scores, but its boosting and bagging models could not contribute to improvement. The LR classifier has better F1-scores than the DT classifier but the boosting and bagging models of the DT classifier do not enhance the F1-scores. In Experiment #4, the RF classifier attained the highest F1-scores, while its boosting and bagging could not improve the scores. The DT classifier achieves better F1-scores than the LR classifier, and its boosting and bagging models help improve the F1-scores. Lastly, in Experiment #5, the LR classifier accomplishes the best F1-scores. The RF classifier ranks second, but its boosting and bagging classifiers could not improve F1 scores. The DT classifier performs well in terms of F1-scores, its bagging classifier helps improve the scores, but it’s boosting one does not contribute to improvement.

61.png

(a)

62.png

(b)

63.png

(c)

64.png

(d)

65.png

(e)

Figure 6. Conventional machine learning classifiers mean of F1-scores of 10 folds of cross-validation

71.png

(a)

72.png

(b)

73.png

(c)

74.png

(d)

75.png

(e)

Figure 7. Distributed machine learning classifiers mean of F1-scores of 10 folds of cross-validation

On the other hand, Figure 7 compares different DML classifiers based on the results of the five experiments conducted on Apache Spark. The DRF classifier achieves the highest F1-scores for all experiments, followed by the DDT classifier in second place. The DFM and DNB classifiers have the lowest F1-scores for different experiments. In Experiment #1, the DLR classifier comes in the third place. The F1-scores of DLSVM and DGBT classifiers are comparable and close to the DLR classifier. In Experiments #2, #3, and #4 the DGBT classifier comes in third place. The DLR classifier shows moderate F1-scores, which are better than the F1-scores of the DLSVM classifier. Lastly, in Experiment #5, the DLR classifier fails in third place. The F1-scores of the DLSVM and DGBT classifiers are similar and close to the DLR classifier.

81.png

(a)

82.png

(b)

83.png

(c)

Figure 8. Random Forest (RF) classifier learning curve

91.png

(a)

92.png

(b)

93.png

(c)

Figure 9. Distributed Random Forest (DFR) classifier accuracy learning curve

4.3.2 Analyzing CML and DML performance via accuracy learning curves

The accuracy learning curves metric provides useful information regarding how a learner can generalize based on the size of the training data. It serves as a useful tool for model selection, predicting the impact of additional training data, and reducing the computational requirements associated with model training and hyperparameter tuning [36, 37]. In most of the experiments discussed in Figure 4 and Figure 5, the RF/DRF classifier demonstrates the best F1-scores. To gain further insights into the performance of the RF and the DRF classifiers, this subsection examines their accuracy learning curves. Figure 8 illustrates the accuracy learning curves of the RF classifier, while Figure 9 depicts the accuracy learning curves of the DRF classifier.

Figure 8 illustrates the accuracy learning curves of the RF classifier. In Figure 8 (a), the training score remains consistently high at 100%, indicating the model's proficiency. On the other hand, the cross-validation score starts at around 96% accuracy rate, highlighting an initial imbalance in the dataset. However, as the training set size increases, the accuracy steadily improves, reaching its peak at approximately 600 and then plateauing after 1000 training samples. The final accuracy on the cross-validation set is approximately 99.68%. Figure 8 (b) focuses on the F1-score learning curve for real job postings. This figure shows that the training score remains at 100% since the entire dataset is used during training. Regarding the cross-validation score, it initially shows a high F1-score of around 97% with a small training set, emphasizing the dataset's imbalance towards real job postings. As the training set size increases, the F1-score is steadily improved, reaching 75% before plateauing after 1000 training samples. Additional data may not significantly enhance the performance of the model. The final evaluation of the cross-validation set yields an impressive F1-score of approximately 99.83%, affirming the model's overall proficiency and effectiveness.

Figure 8 (c) represents the F1-score learning curve for fake job postings. Similar to the previous figures, the training score remains at 100%. According to the cross-validation score, the initial F1-score is around 70% with a limited training set and it is steadily improved as the training set size increases. The accuracy peaks at approximately 600 and plateaus after 1250 training samples. Additional data may not yield significant enhancements in the performance of the model. The final evaluation of the cross-validation set results in an impressive F1-score of approximately 96.99%, indicating commendable overall performance.

Moving on to Figure 9, it demonstrates the accuracy learning curves of the DRF classifier. Similar to the RF classifier, the training score remains high at 100% as shown in Figure 9 (a). As regards the cross-validation score, the accuracy starts at around 90% with a small training set but it is improved steadily as the training set size increases. The accuracy reaches its peak at approximately 500 and then plateaus after 550 training samples, suggesting that additional data might not significantly enhance the performance of the model. The final accuracy on the cross-validation set is approximately 97.58%.

Figure 9 (b) concentrates on the F1-score learning curve for real job postings with the DRF classifier. Like the RF classifier, the model's training score remains consistently 100%. As regards the cross-validation score, the initial cross-validation phase yields an F1-score of approximately 94% with a small training set, which increases steadily to around 94.5% as the training set size grows. The learning curve plateaus after 500 training samples, indicating that the additional data might not yield significant improvements in the model's performance. The final evaluation of the cross-validation set results in an F1-score of approximately 98.67%, indicating a well-performing model.

Lastly, Figure 9 (c) depicts the F1-score learning curve for fake job postings with the DRF classifier. Like the RF classifier, the training score remains consistently 100%. Regarding the cross-validation score, the initial cross-validation phase with a small training set yields an F1-score of around 60%, which increases steadily to approximately 80% as the training set size grows. The learning curve plateaus after 500 training samples, suggesting that additional data might not significantly enhance the model's performance. The final evaluation of the cross-validation set results in an F1-score of approximately 86.70%, indicating a reasonable performance by the model.

4.4 Discussion of results

Tables 6-9 summarize the results of the above five experiments for both CML and DML classifiers. Based on the findings presented in Tables 6-9 for both CML and DML classifiers, it is evident that the Random Forest (RF) classifier is the most effective classifier in the CML category. Throughout all five experiments, RF consistently achieves high precision, recall, and F1-scores for both classes (0 and 1), along with impressive accuracy rates. Meanwhile, in the DML category, The Distributed Random Forest (DRF) classifier is the top performer. DRF demonstrates superior F1-scores in most experiments and outperforms other classifiers, including the Decision Tree (DDT) when considering the mean F1-scores from 10-fold cross-validation. Notably, DRF shows robust performance in terms of F1-score for class 1 and accuracy, making it the preferred choice for DML tasks. Generally, while RF excels in CML settings, DRF proves to be the go-to classifier for DML, showcasing the importance of selecting the appropriate model based on the specific context and task requirements.

The high accuracy and F1-scores suggest that the framework can effectively detect fake job postings, improving user security and trust across known platforms like LinkedIn and Indeed. Its adaptability to handle imbalanced data, such as the 95% real and 5% fake job ratio, is critical in real-time operations where fraudulent postings are less common but harmful. The scalability provided by Distributed Machine Learning (DML) models based on big data analytics, allows the framework to handle high traffic on large platforms, making it suitable for platforms with a significant volume and velocity of job postings. Furthermore, the feedback loop enables continuous learning, ensuring that the framework evolves with emerging scam tactics, thus offering long-term protection. This could lead to substantial economic benefits for job platforms by improving their reputation and user retention, while also enhancing user safety by reducing the risk of fraud. However, real-world deployment might face challenges such as false positives or negatives, which need further refinement to minimize errors. Future research should focus on testing the framework across different platforms and datasets to ensure its generalizability and robustness in diverse environments.

Table 6. CML classifiers scores

Table 7. CML classifiers mean of F1-score of 10 folds of cross-validation

Table 8. DML classifiers scores

Table 9. DML classifiers mean of F1-score of 10 folds of cross-validation