Machine Learning, Regression and Optimization

Machine learning is a subfield of artificial intelligence (AI). While AI is the ability of the machine to think like humans, machine learning is the ability of machine to learn from data without any explicit instructions. Applications of machine learning are abundant: stock-price forecast; face, speech and handwriting recognition; medical diagnosis of diseases like cancer, blood pressure, diabetes, neurological disorders including autism, spinal stenosis and others; and health monitoring, just to name a few. Potential applications of machine learning in solutions to many other complex practical problems are currently being investigated. An ultimate goal of machine learning is to make predictions based on a properly trained model. Two major techniques of supervised machine learning are: statistical regression and classification. For best prediction, the parameters of the model need to be optimized. This is an optimization task. After giving a brief introduction to machine learning and describing the role of regression and optimization, the paper discusses in some detail the basics of regression and optimization methods that are commonly used in machine learning. The paper is interdisciplinary, blending machine learning with statistical regression and numerical linear algebra, and optimization. Thus, it will be of interest to a wide variety of audiences ranging from mathematics, statistics and computer science to various branches of engineering.