IDEAS home Printed from https://ideas.repec.org/a/bla/polstu/v57y2009i2p356-373.html
   My bibliography  Save this article

Design Experiments: Engaging Policy Makers in the Search for Evidence about What Works

Author

Listed:
  • Gerry Stoker
  • Peter John

Abstract

This article presents an argument for the greater use of design experiments, which can assist policy making because they provide both robust and timely evidence. We discuss their origins in education research, set out the methodology and propose some adaptations to the techniques used in these education studies to foster their application to a range of policy fields and problem areas. Design experiments need to meet two challenges. Can they provide valid evidence? Can they provide evidence that will be used by policy makers? Our argument shows how design experiments are robust when set against the classical canons of scientific study. We further claim that the design experiment approach offers a more viable means to developing evidence‐based policy making than other forms of evaluation because of the timeliness of the insights that it provides.

Suggested Citation

  • Gerry Stoker & Peter John, 2009. "Design Experiments: Engaging Policy Makers in the Search for Evidence about What Works," Political Studies, Political Studies Association, vol. 57(2), pages 356-373, June.
  • Handle: RePEc:bla:polstu:v:57:y:2009:i:2:p:356-373
    DOI: 10.1111/j.1467-9248.2008.00756.x
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/j.1467-9248.2008.00756.x
    Download Restriction: no

    File URL: https://libkey.io/10.1111/j.1467-9248.2008.00756.x?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Herbert A. Simon, 1996. "The Sciences of the Artificial, 3rd Edition," MIT Press Books, The MIT Press, edition 1, volume 1, number 0262691914, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Claire A Dunlop, 2014. "The Possible Experts: How Epistemic Communities Negotiate Barriers to Knowledge Use in Ecosystems Services Policy," Environment and Planning C, , vol. 32(2), pages 208-228, April.
    2. Warren Pearce & Sujatha Raman, 2014. "The new randomised controlled trials (RCT) movement in public policy: challenges of epistemic governance," Policy Sciences, Springer;Society of Policy Sciences, vol. 47(4), pages 387-402, December.
    3. Laurent Hazard & Nathalie Couix & Camille Lacombe, 2022. "From evidence to value-based transition: the agroecological redesign of farming systems," Agriculture and Human Values, Springer;The Agriculture, Food, & Human Values Society (AFHVS), vol. 39(1), pages 405-416, March.
    4. Eva Sørensen & Jacob Torfing, 2021. "Accountable Government through Collaborative Governance?," Administrative Sciences, MDPI, vol. 11(4), pages 1-20, November.
    5. Ansell, Christopher K. & Bartenberger, Martin, 2016. "Varieties of experimentalism," Ecological Economics, Elsevier, vol. 130(C), pages 64-73.
    6. Gerry Stoker, 2010. "Exploring the Promise of Experimentation in Political Science: Micro‐Foundational Insights and Policy Relevance," Political Studies, Political Studies Association, vol. 58(2), pages 300-319, March.
    7. Ringa Raudla & Külli Sarapuu & Johanna Vallistu & Kerli Onno & Nastassia Harbuzova, 2025. "The politics of experimental policymaking: the influence of blame avoidance and credit claiming," Policy Sciences, Springer;Society of Policy Sciences, vol. 58(2), pages 245-266, June.
    8. Gerry Stoker, 2010. "Translating Experiments into Policy," The ANNALS of the American Academy of Political and Social Science, , vol. 628(1), pages 47-58, March.
    9. Broström, Anders & McKelvey, Maureen, 2016. "Knowledge transfer at the science-policy interface: How cognitive distance and the degree of expert autonomy shapes the outcome," Working Paper Series in Economics and Institutions of Innovation 441, Royal Institute of Technology, CESIS - Centre of Excellence for Science and Innovation Studies.
    10. Steven Donbavand & Bryony Hoskins, 2021. "Citizenship Education for Political Engagement: A Systematic Review of Controlled Trials," Social Sciences, MDPI, vol. 10(5), pages 1-19, April.
    11. Guillaume Martin & Sandrine Allain & Jacques-Eric Bergez & Delphine Burger-Leenhardt & Julie Constantin & Michel Duru & Laurent Hazard & Camille Lacombe & Danièle Magda & Marie-Angélina Magne & Julie , 2018. "How to Address the Sustainability Transition of Farming Systems? A Conceptual Framework to Organize Research," Sustainability, MDPI, vol. 10(6), pages 1-20, June.
    12. Shihong Guo & Qijiao Song & Ye Qi, 2021. "Innovation or implementation? Local response to low‐carbon policy experimentation in China," Review of Policy Research, Policy Studies Organization, vol. 38(5), pages 555-569, September.
    13. Michael McGann & Emma Blomkamp & Jenny M. Lewis, 2018. "The rise of public sector innovation labs: experiments in design thinking for policy," Policy Sciences, Springer;Society of Policy Sciences, vol. 51(3), pages 249-267, September.
    14. Eric Chu & Todd Schenk & James Patterson, 2018. "The Dilemmas of Citizen Inclusion in Urban Planning and Governance to Enable a 1.5 °C Climate Change Scenario," Urban Planning, Cogitatio Press, vol. 3(2), pages 128-140.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Tobias Knabke & Sebastian Olbrich, 2018. "Building novel capabilities to enable business intelligence agility: results from a quantitative study," Information Systems and e-Business Management, Springer, vol. 16(3), pages 493-546, August.
    2. Sunder Shyam, 2011. "Imagined Worlds of Accounting," Accounting, Economics, and Law: A Convivium, De Gruyter, vol. 1(1), pages 1-14, January.
    3. McCown, R. L., 2002. "Changing systems for supporting farmers' decisions: problems, paradigms, and prospects," Agricultural Systems, Elsevier, vol. 74(1), pages 179-220, October.
    4. Basile, Luigi Jesus & Carbonara, Nunzia & Pellegrino, Roberta & Panniello, Umberto, 2023. "Business intelligence in the healthcare industry: The utilization of a data-driven approach to support clinical decision making," Technovation, Elsevier, vol. 120(C).
    5. Loris Gaio, 2005. "A diversity-based approach to requirements tracing in new product development," ROCK Working Papers 031, Department of Computer and Management Sciences, University of Trento, Italy, revised 13 Jun 2008.
    6. B. A. Huberman & N. S. Glance, "undated". "Diversity and Collective Action," Working Papers _001, Xerox Research Park.
    7. Zhewei Zhang & Youngjin Yoo & Kalle Lyytinen & Aron Lindberg, 2021. "The Unknowability of Autonomous Tools and the Liminal Experience of Their Use," Information Systems Research, INFORMS, vol. 32(4), pages 1192-1213, December.
    8. David Stadelmann & Benno Torgler, 2012. "Bounded Rationality and Voting Decisions Exploring a 160-Year Period," Working Papers 2012.70, Fondazione Eni Enrico Mattei.
    9. Francis Marleau Donais & Irène Abi-Zeid & E. Owen D. Waygood & Roxane Lavoie, 2021. "A Framework for Post-Project Evaluation of Multicriteria Decision Aiding Processes from the Stakeholders’ Perspective: Design and Application," Group Decision and Negotiation, Springer, vol. 30(5), pages 1161-1191, October.
    10. H. Christopher Frey & Sumeet R. Patil, 2002. "Identification and Review of Sensitivity Analysis Methods," Risk Analysis, John Wiley & Sons, vol. 22(3), pages 553-578, June.
    11. Marie-Laure Salles-Djelic & Michel Gutsatz, 2000. "Managerial Competencies for Organizational Flexibility: The Luxury Goods Industry between Tradition and Postmodernism," Post-Print hal-01892018, HAL.
    12. Rennard, Jean-Philippe, 2006. "Artificiality in Social Sciences," MPRA Paper 1458, University Library of Munich, Germany.
    13. Luoma, Jukka, 2016. "Model-based organizational decision making: A behavioral lens," European Journal of Operational Research, Elsevier, vol. 249(3), pages 816-826.
    14. Dalila Cisco Collatto & Aline Dresch & Daniel Pacheco Lacerda & Ione Ghislene Bentz, 2018. "Is Action Design Research Indeed Necessary? Analysis and Synergies Between Action Research and Design Science Research," Systemic Practice and Action Research, Springer, vol. 31(3), pages 239-267, June.
    15. Nadia Fiorino & Emma Galli & Ilde Rizzo & Marco Valente, 2023. "Public procurement and reputation. An agent‐based model," Metroeconomica, Wiley Blackwell, vol. 74(4), pages 806-832, November.
    16. Olivier L. de Weck & Marshall B. Jones, 2006. "Isoperformance: Analysis and design of complex systems with desired outcomes," Systems Engineering, John Wiley & Sons, vol. 9(1), pages 45-61, March.
    17. Hippel, Eric von., 1992. "Adapting market research to the rapid evolution of needs for new products and services," Working papers 3374-92., Massachusetts Institute of Technology (MIT), Sloan School of Management.
    18. Konstantinos S. Boulas & Georgios D. Dounias & Chrissoleon T. Papadopoulos, 2023. "A hybrid evolutionary algorithm approach for estimating the throughput of short reliable approximately balanced production lines," Journal of Intelligent Manufacturing, Springer, vol. 34(2), pages 823-852, February.
    19. Richard Fellows & Anita M.M. Liu, 2012. "Managing organizational interfaces in engineering construction projects: addressing fragmentation and boundary issues across multiple interfaces," Construction Management and Economics, Taylor & Francis Journals, vol. 30(8), pages 653-671, February.
    20. Tay, Nicholas S.P. & Lusch, Robert F., 2005. "A preliminary test of Hunt's General Theory of Competition: using artificial adaptive agents to study complex and ill-defined environments," Journal of Business Research, Elsevier, vol. 58(9), pages 1155-1168, September.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:polstu:v:57:y:2009:i:2:p:356-373. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: http://www.blackwellpublishing.com/journal.asp?ref=0032-3217 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.