Abstract
This study evaluates the effectiveness of transformer-based deep learning models in improving credit risk assessment for predicting default probabilities among credit card customers. By employing the CNN-SFTransformer and GRU-Transformer models, this research aims to enhance predictive accuracy and robustness compared to traditional machine learning methods. The models were trained and tested on diverse datasets from Taiwan, Germany, and Australia, representing various credit risk scenarios. The experimental setup included rigorous hyperparameter tuning and utilized key evaluation metrics such as ROC AUC, KS statistic, and G-˜ µ to assess model performance comprehensively. The CNN-SFTransformer model demonstrated superior performance , consistently surpassing baseline models like LSTM, Support Vector Machines (SVM), and Random Forest across all datasets. This performance indicates its enhanced capability in differentiating between defaulters and non-defaulters. The GRU-Transformer model also showed promising results, further validating the effectiveness of transformer ar-chitectures in this domain. Statistical significance of the results was confirmed through the McNemar test, ensuring the robustness and reliability of the proposed models. This research introduces a novel approach to credit risk management by providing scalable and adaptable models that improve the precision of default predictions, thereby aiding financial institutions in making more informed lending decisions with greater confidence.