Massaging Significance in the Age of Open Science: A Satirical Tutorial

The open science movement has made life considerably harder for researchers committed to the traditional craft of producing tidy, statistically significant stories from inconveniently untidy data that the publishing system still finds deeply attractive. Preregistration, data sharing, Registered Reports, and growing expectations of reproducibility have introduced unwelcome friction into once-fluid research workflows. In this satirical tutorial, we offer a service to the creatively empirical by documenting how significance may still be massaged in an era increasingly hostile to methodological improvisation. We identify three broad strategies. First, preregistration can be used less as a constraint than as a menu, allowing vagueness, selective reporting, and preregistration forks to preserve analytic flexibility. Second, exploratory findings can be dressed in confirmatory clothing by leaving the boundary between the two unmarked. Third, open science can be practiced opaquely, such that materials, data, and codes are technically available while remaining functionally unreachable. We then describe emerging threats to these strategies, including stricter preregistration standards, reproducibility checks, and mandatory study registration. Reassuringly for the creatively empirical, most of these reforms remain unlikely, as they would require meaningful changes to publishing incentives and coordinated effort among scientists, universities, and journals that academia rarely produces except in mission statements.