Abstract
This study evaluates the effectiveness of transformer-based deep learning models in improving credit risk assessment for predicting default probabilities among credit card customers. By employing the Convolutional Neural Network Semantic Feature Transformer (CNN-SFTransformer) and Gated Recurrent Unit Transformer (GRU-Transformer) models, this study aims to enhance predictive accuracy and robustness compared to traditional machine learning methods. The models were trained and tested on diverse datasets from Taiwan, Germany, and Australia, representing various credit risk scenarios. The experimental setup included hyperparameter tuning and utilized evaluation metrics such as ROC AUC (Receiver Operating Characteristic Area Under the Curve), KS statistic (Kolmogorov-Smirnov), and G-𝜇 ̃(Geometric mean) to assess model performance. The CNN-SFTransformer model demonstrated superior performance, consistently surpassing baseline models such as Long Short-Term Memory (LSTM), Support Vector Machine (SVM), and Random Forest (RF) across all datasets. This performance indicates CNN-SFTransformormer capability in differentiating between defaulters and non-defaulters. The GRU-Transformer model also showed promising results, further validating the effectiveness of transformer architectures in this domain. Statistical significance of the results was confirmed through the McNemar test, ensuring the robustness and reliability of the proposed models. This study introduces a novel approach to credit risk management by providing scalable and adaptable models that improve default prediction accuracy, aiding financial institutions in making more informed lending decisions.