Logistic Regression

This chapter covers logistic regression, which is a widely used method in analytics projects for predicting binary outcomes. The chapter begins by explaining the difference between ordinary linear and logistic regression when dealing with binary outcomes. Logistic regression is preferred for binary targets as it provides predictions ranging from 0.0 to 1.0, representing the probability of the target variable taking on the value of 1. The chapter introduces the logistic function and discusses analyses with binary outcomes. It also explores the metrics used to assess predictive models with binary or multi-level categorical targets, relevant for later chapters covering other prediction models. The logistic model is demonstrated with examples using simulated data and real-world data related to employee turnover and heart disease prediction. The importance of interpreting coefficients in logistic regression is discussed, and various approaches to interpreting predictors and assessing model performance are explored, including confusion matrices and ROC curves. The chapter also covers applying regularization techniques (L1 and L2 regularization) to logistic regression models to improve generalizability and mitigate overfitting. The concept of asymmetric costs and benefits in predictive models is introduced, particularly in the context of medical applications. Finally, the chapter introduces multinomial logistic regression for cases where the target variable has more than two categorical levels. An example using the Iris data set is provided to demonstrate the multinomial logistic regression approach. Overall, this chapter provides a comprehensive overview of logistic regression, its interpretation, performance evaluation, regularization, and its extension to multinomial cases. It offers valuable insights for data analysts and researchers working with binary and multi-level categorical outcomes in their predictive modeling tasks.