Are mortgage loan closing delay risks predictable? A predictive analysis using text mining on discussion threads

Loan processors and underwriters at mortgage firms seek to gather substantial supporting documentation to properly understand and model loan risks. In doing so, loan originations become prone to closing delays, risking client dissatisfaction and consequent revenue losses. We collaborate with a large national mortgage firm to examine the extent to which these delays are predictable, using internal discussion threads to prioritize interventions for loans most at risk. Substantial work experience is required to predict delays, and we find that even highly trained employees have difficulty predicting delays by reviewing discussion threads. We develop an array of methods to predict loan delays. We apply four modern out‐of‐the‐box sentiment analysis techniques, two dictionary‐based and two rule‐based, to predict delays. We contrast these approaches with domain‐specific approaches, including firm‐provided keyword searches and “smoke terms” derived using machine learning. Performance varies widely across sentiment approaches; while some sentiment approaches prioritize the top‐ranking records well, performance quickly declines thereafter. The firm‐provided keyword searches perform at the rate of random chance. We observe that the domain‐specific smoke term approaches consistently outperform other approaches and offer better prediction than loan and borrower characteristics. We conclude that text mining solutions would greatly assist mortgage firms in delay prevention.