IDEAS home Printed from https://ideas.repec.org/a/bla/polstu/v57y2009i2p356-373.html
   My bibliography  Save this article

Design Experiments: Engaging Policy Makers in the Search for Evidence about What Works

Author

Listed:
  • Gerry Stoker
  • Peter John

Abstract

This article presents an argument for the greater use of design experiments, which can assist policy making because they provide both robust and timely evidence. We discuss their origins in education research, set out the methodology and propose some adaptations to the techniques used in these education studies to foster their application to a range of policy fields and problem areas. Design experiments need to meet two challenges. Can they provide valid evidence? Can they provide evidence that will be used by policy makers? Our argument shows how design experiments are robust when set against the classical canons of scientific study. We further claim that the design experiment approach offers a more viable means to developing evidence‐based policy making than other forms of evaluation because of the timeliness of the insights that it provides.

Suggested Citation

  • Gerry Stoker & Peter John, 2009. "Design Experiments: Engaging Policy Makers in the Search for Evidence about What Works," Political Studies, Political Studies Association, vol. 57(2), pages 356-373, June.
  • Handle: RePEc:bla:polstu:v:57:y:2009:i:2:p:356-373
    DOI: 10.1111/j.1467-9248.2008.00756.x
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/j.1467-9248.2008.00756.x
    Download Restriction: no

    File URL: https://libkey.io/10.1111/j.1467-9248.2008.00756.x?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Herbert A. Simon, 1996. "The Sciences of the Artificial, 3rd Edition," MIT Press Books, The MIT Press, edition 1, volume 1, number 0262691914, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Claire A Dunlop, 2014. "The Possible Experts: How Epistemic Communities Negotiate Barriers to Knowledge Use in Ecosystems Services Policy," Environment and Planning C, , vol. 32(2), pages 208-228, April.
    2. Warren Pearce & Sujatha Raman, 2014. "The new randomised controlled trials (RCT) movement in public policy: challenges of epistemic governance," Policy Sciences, Springer;Society of Policy Sciences, vol. 47(4), pages 387-402, December.
    3. Laurent Hazard & Nathalie Couix & Camille Lacombe, 2022. "From evidence to value-based transition: the agroecological redesign of farming systems," Agriculture and Human Values, Springer;The Agriculture, Food, & Human Values Society (AFHVS), vol. 39(1), pages 405-416, March.
    4. Eva Sørensen & Jacob Torfing, 2021. "Accountable Government through Collaborative Governance?," Administrative Sciences, MDPI, vol. 11(4), pages 1-20, November.
    5. Ansell, Christopher K. & Bartenberger, Martin, 2016. "Varieties of experimentalism," Ecological Economics, Elsevier, vol. 130(C), pages 64-73.
    6. Gerry Stoker, 2010. "Exploring the Promise of Experimentation in Political Science: Micro‐Foundational Insights and Policy Relevance," Political Studies, Political Studies Association, vol. 58(2), pages 300-319, March.
    7. Gerry Stoker, 2010. "Translating Experiments into Policy," The ANNALS of the American Academy of Political and Social Science, , vol. 628(1), pages 47-58, March.
    8. Broström, Anders & McKelvey, Maureen, 2016. "Knowledge transfer at the science-policy interface: How cognitive distance and the degree of expert autonomy shapes the outcome," Working Paper Series in Economics and Institutions of Innovation 441, Royal Institute of Technology, CESIS - Centre of Excellence for Science and Innovation Studies.
    9. Steven Donbavand & Bryony Hoskins, 2021. "Citizenship Education for Political Engagement: A Systematic Review of Controlled Trials," Social Sciences, MDPI, vol. 10(5), pages 1-19, April.
    10. Guillaume Martin & Sandrine Allain & Jacques-Eric Bergez & Delphine Burger-Leenhardt & Julie Constantin & Michel Duru & Laurent Hazard & Camille Lacombe & Danièle Magda & Marie-Angélina Magne & Julie , 2018. "How to Address the Sustainability Transition of Farming Systems? A Conceptual Framework to Organize Research," Sustainability, MDPI, vol. 10(6), pages 1-20, June.
    11. Shihong Guo & Qijiao Song & Ye Qi, 2021. "Innovation or implementation? Local response to low‐carbon policy experimentation in China," Review of Policy Research, Policy Studies Organization, vol. 38(5), pages 555-569, September.
    12. Michael McGann & Emma Blomkamp & Jenny M. Lewis, 2018. "The rise of public sector innovation labs: experiments in design thinking for policy," Policy Sciences, Springer;Society of Policy Sciences, vol. 51(3), pages 249-267, September.
    13. Eric Chu & Todd Schenk & James Patterson, 2018. "The Dilemmas of Citizen Inclusion in Urban Planning and Governance to Enable a 1.5 °C Climate Change Scenario," Urban Planning, Cogitatio Press, vol. 3(2), pages 128-140.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Alan Hevner & Isabelle Comyn-Wattiau & Jacky Akoka & Nicolas Prat, 2018. "A pragmatic approach for identifying and managing design science research goals and evaluation criteria," Post-Print hal-02283783, HAL.
    2. Tobias Knabke & Sebastian Olbrich, 2018. "Building novel capabilities to enable business intelligence agility: results from a quantitative study," Information Systems and e-Business Management, Springer, vol. 16(3), pages 493-546, August.
    3. Sunder Shyam, 2011. "Imagined Worlds of Accounting," Accounting, Economics, and Law: A Convivium, De Gruyter, vol. 1(1), pages 1-14, January.
    4. Fiori Stefano, 2005. "The emergence of instructions : some open problems in Hayek's theory," CESMEP Working Papers 200504, University of Turin.
    5. McCown, R. L., 2002. "Changing systems for supporting farmers' decisions: problems, paradigms, and prospects," Agricultural Systems, Elsevier, vol. 74(1), pages 179-220, October.
    6. Jin P. Gerlach & Ronald T. Cenfetelli, 2022. "Overcoming the Single-IS Paradigm in Individual-Level IS Research," Information Systems Research, INFORMS, vol. 33(2), pages 476-488, June.
    7. Basile, Luigi Jesus & Carbonara, Nunzia & Pellegrino, Roberta & Panniello, Umberto, 2023. "Business intelligence in the healthcare industry: The utilization of a data-driven approach to support clinical decision making," Technovation, Elsevier, vol. 120(C).
    8. Loris Gaio, 2005. "A diversity-based approach to requirements tracing in new product development," ROCK Working Papers 031, Department of Computer and Management Sciences, University of Trento, Italy, revised 13 Jun 2008.
    9. B. A. Huberman & N. S. Glance, "undated". "Diversity and Collective Action," Working Papers _001, Xerox Research Park.
    10. Zhewei Zhang & Youngjin Yoo & Kalle Lyytinen & Aron Lindberg, 2021. "The Unknowability of Autonomous Tools and the Liminal Experience of Their Use," Information Systems Research, INFORMS, vol. 32(4), pages 1192-1213, December.
    11. Juval Portugali & Egbert Stolk, 2014. "A SIRN View on Design Thinking—An Urban Design Perspective," Environment and Planning B, , vol. 41(5), pages 829-846, October.
    12. Gilbert Giacomoni & Adel Aloui, 2018. "Imaginaire et imitation du réel : genèse des idées et sciences de l’artificiel," Post-Print hal-01941661, HAL.
    13. Funk, Jeffery, 2009. "Components, systems and discontinuities: The case of magnetic recording and playback equipment," Research Policy, Elsevier, vol. 38(7), pages 1192-1202, September.
    14. Richard Holt & J. Barkley Rosser & David Colander, 2011. "The Complexity Era in Economics," Review of Political Economy, Taylor & Francis Journals, vol. 23(3), pages 357-369.
    15. David Stadelmann & Benno Torgler, 2012. "Bounded Rationality and Voting Decisions Exploring a 160-Year Period," Working Papers 2012.70, Fondazione Eni Enrico Mattei.
    16. Karén Hovhannissian & Marco Valente, 2004. "Modeling Directed Local Search Strategies on Technology Landscapes: Depth and Breadth," ROCK Working Papers 028, Department of Computer and Management Sciences, University of Trento, Italy, revised 17 Jun 2008.
    17. Pan Guo & Xiaofeng Li & Yanlin Jia & Xu Zhang, 2020. "Cloud Model-Based Comprehensive Evaluation Method for Entrepreneurs’ Uncertainty Tolerance," Mathematics, MDPI, vol. 8(9), pages 1-14, September.
    18. Francis Marleau Donais & Irène Abi-Zeid & E. Owen D. Waygood & Roxane Lavoie, 2021. "A Framework for Post-Project Evaluation of Multicriteria Decision Aiding Processes from the Stakeholders’ Perspective: Design and Application," Group Decision and Negotiation, Springer, vol. 30(5), pages 1161-1191, October.
    19. H. Christopher Frey & Sumeet R. Patil, 2002. "Identification and Review of Sensitivity Analysis Methods," Risk Analysis, John Wiley & Sons, vol. 22(3), pages 553-578, June.
    20. Marie-Laure Salles-Djelic & Michel Gutsatz, 2000. "Managerial Competencies for Organizational Flexibility: The Luxury Goods Industry between Tradition and Postmodernism," Post-Print hal-01892018, HAL.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:polstu:v:57:y:2009:i:2:p:356-373. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: http://www.blackwellpublishing.com/journal.asp?ref=0032-3217 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.