Theoretical aspects of statistical analysis of unstructured data from chatbots for improving customer experience

In today's digitalized world, one of the computer systems that saves a lot of time and effort for both users and the company is the chatbot. It facilitates customer service through automation and at the same time creates a large volume of high-dimensional, unstructured text data. This report aims to create a theoretical and methodological framework for statistical analysis of chatbot conversations by transforming unstructured dialogue into statistically measurable quantities for the needs of economic analysis. Since traditional statistical methods are not sufficient to extract economic value from this type of text to improve the way customers are served, an approach was applied that focuses on advanced natural language processing (NLP) methods such as: tokenization, lemmatization, stop words, vectorization, sentiment analysis, and thematic modeling. Inferential statistics and predictive models were also applied to understand whether the change in the chatbot really leads to user satisfaction and to predict how it will be in the future. It also sheds light on how analyzing such data improves the customer experience.