The Role of the Third Sector in Public Health Service Provision: Evidence from 25,338 heterogeneous procurement datasets

The role of external suppliers across statutory health insurance procurement processes varies widely and is a source of political contention throughout the modern world. We comprehensively examine the role of non-profit organisations in public health procurement within publicly funded healthcare which runs parallel to private provision in a 'two-tier' system. We build a unique 'Big Data' based pipeline which scrapes tens of thousands of heterogeneous accounting datasets from across a commissioning hierarchy. These datasets provide granular information on every element of procurement at the micro-level (where the value of a transaction is greater than twenty-five thousand pounds), mandated by transparency requirements introduced by David Cameron in 2010. We develop tools to scrape, parse, and reconcile suppliers with institutional registers. The processed dataset contains over four hundred and forty-five billion pounds worth of commissioning across over 1.9 million rows of clean data. Approximately 1% at each level of procurement comes from institutions listed on the Charity Commission for England and Wales: a number relatively consistent across time, despite contractual patterns. We show a slight regional variation and analyse the 'North-South' divide. Linking to the International Classification of Non-profit Organizations, we show involvement of multiple different types of charity, with more payments going to the 'Social Services' aggregate, but the highest cumulative values going to the 'Health' aggregate. We analyse the distribution across various sizes and ages, from grassroots to 'Super Major' non-profits, and analyse variation over time. We conclude with a re-evaluation of the effects of the controversial Health and Social Care Act of 2012 and the integration of the free market and volunteerism, otherwise known as the 'Big Society'.