Bayesian Audit Outcome Model Selection Using Normalizing Flows

The capability to predict audit opinions is important for auditors concerning audit risk rating and resource allocation. Machine learning has successfully been deployed to model audit opinions for public and listed entities in various jurisdictions. Given the proliferation of machine learning models for this task, selecting the most appropriate model for modeling audit opinions for a given dataset has become crucial. This chapter uses the Bayesian evidence metric to select a model for modeling audit outcomes in South African municipalities. Our framework approximates the evidence using a learned harmonic mean evidence estimator based on normalizing flows. The normalizing flows-based harmonic mean evidence estimator utilizes samples generated from the Metropolis-Adjusted Langevin Algorithm, Hamiltonian Monte Carlo, and No-U-Turn Sampler Markov Chain Monte Carlo methods to learn the target distribution. We compare models based on the Bayesian logistic regression model but with different financial ratios (covariate) configurations. Our results show that the Bayesian evidence framework can reliably predict which model will outperform on unseen data based on the evidence metric calculated on training data, and we find a positive correlation between evidence on training data and the Area Under the Receiver Operating Curve metric on unseen data. This framework can be easily extended to encapsulate a broader class of models, such as for optimal architecture search for Bayesian neural networks, albeit at an increased computational cost.