From Singing in the Rain to Tears in the Rain: Socio-demographic Trends and Pessimism during New Hollywood

New Hollywood (NH) was a transformative period in American cinema from 1967 to 1982, marked by a shift towards more experimental, morally ambiguous, and pessimistic storytelling. Film studies surmise these shifts were influenced by socio-economic challenges such as declining movie revenues, stagnating income, rising distrust in institutions, and increased college-educated audiences. However, it is unclear whether these factors generally lead to ambiguous and pessimistic storytelling. In this study, we use Natural Language Processing (NLP) to quantify psychological dimensions of pessimism, ambiguity, stress, and negativity across 5,948 movie scripts spanning 1950-2000 and comprising the periods NH (1967-1982), pre-NH (<1967) and post-NH (>1982). We contrasted the psychological dynamics across periods and assessed the temporal precedence of socioeconomic variables using lagged regression, cross-correlation, and network analysis. First, we confirmed that pessimism, stress, and ambiguity increased during NH, while pessimism and ambiguity decreased afterward. However, negativity consistently rose throughout 1950-2000. Second, declining trust in institutions and movie revenues predicted higher levels of pessimism and ambiguity, with college education correlating with lower pessimism and ambiguity but higher negativity and stress. Third, median income and income growth had complex effects: pessimism, negativity, and stress correlated with a decline in median income growth but a rise in median income. Finally, a decline in income growth temporally preceded a significant increase in pessimism, suggesting a lagged effect of economic downturns on cultural expression. The study provides a framework for understanding how socioeconomic conditions influence cinematic content, emphasizing the complex interplay between material conditions and cultural outputs.