Mitigating adversarial attacks on transformer models in credit scoring

The integration of unstructured data, such as text created by borrowers, offers new opportunities for improving credit default prediction but also introduces new risks. This study examines the robustness of transformer-based credit scoring models that utilize textual data and assesses their vulnerability to adversarial attacks. Using peer-to-peer lending data, we show that small, semantically neutral changes in loan descriptions can substantially alter model outputs. These vulnerabilities expose lenders and borrowers to economic risks through distorted risk assessments and mispriced loans. We evaluate two mitigation strategies: adversarial training and topic modeling. Adversarial training improves robustness without compromising predictive performance. Topic modeling provides a more interpretable and stable representation of borrower narratives. An economic analysis confirms that robust models reduce mispricing and improve outcomes for all parties. The findings underscore the importance of robustness as the use of unstructured data in credit scoring becomes more accessible.