Scaling Product Acceptance Analysis Using Annotation Systems Based on Large Language Models

While Technology Acceptance Models (TAM) are widely used, their practical utility remains in question. Significant challenges to the useful application of TAM include the difficulties in gathering TAM data through traditional methods of surveys and questionnaires and the homogeneity hypothesis for technology predictors, which constrains the usefulness of TAM models in comparing products. This paper addresses both challenges by (1) validating the use of NLP-LLM to incorporate unstructured online data into TAM models rather than surveys and (2) demonstrating and validating a TAM model without homogenous predictors by applying it to the handheld gaming console market. We use Amazon reviews for Nintendo, Steam Deck, Logitech, and Razer consoles, annotating them using Chat GPT NLP-LLM-algorithms to obtain scores for TAM variables. We validate the use of NLP-LLMs by measuring high consistency between LLM runs and strong agreement with scores given by human experts. We then fit separate TAM regressions for each console and compare coefficients, revealing significant heterogeneity with the Steam Deck emerging as a market leader and Nintendo lagging behind its competitors. Altogether, we demonstrate that TAM models with heterogeneous predictors are both useful and applicable, and that NLP-LLM powered models have the ability to provide results highly similar to current more expensive and limited methods.