IDEAS home Printed from https://ideas.repec.org/a/sae/evarev/v40y2016i1p29-60.html

Methodological Credibility

Author

Listed:
  • Miriam R. Jacobson
  • Tarek Azzam

Abstract

Background: When evaluations are broadly disseminated, the public can use them to support a program or to advocate for change. Methods: To explore how evaluations are perceived and used by the public, individuals in a sample of 425 people in the United States were recruited through an online crowdsourcing service called Mechanical Turk ( www.mturk.com ). Participants were randomly assigned to receive different versions of a press release describing a summative evaluation of a program. Each condition contained a unique combination of methods (e.g., randomized controlled design) and findings (positive or negative) to describe the evaluation and its findings. Participants in each condition responded to questions about their trust in the content of the evaluation findings and their attitudes toward the program. Results: Results indicated that the type of evaluation methods and the direction of the findings both influenced the credibility of the findings and that the credibility of the findings moderated the relationship between the direction of the evaluation findings and attitudes toward the evaluated program. Additional evaluation factors to explore in future research with the public are recommended.

Suggested Citation

  • Miriam R. Jacobson & Tarek Azzam, 2016. "Methodological Credibility," Evaluation Review, , vol. 40(1), pages 29-60, February.
    • Handle: RePEc:sae:evarev:v:40:y:2016:i:1:p:29-60
    DOI: 10.1177/0193841X16657728
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/0193841X16657728
    Download Restriction: no
    File URL: https://libkey.io/10.1177/0193841X16657728?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Daigneault, Pierre-Marc, 2014. "Taking stock of four decades of quantitative research on stakeholder participation and evaluation use: A systematic map," Evaluation and Program Planning, Elsevier, vol. 45(C), pages 171-181.
    2. Richard T. Carson, 2012. "Contingent Valuation: A Practical Alternative When Prices Aren't Available," Journal of Economic Perspectives, American Economic Association, vol. 26(4), pages 27-42, Fall.
    3. Azzam, Tarek & Levine, Bret, 2015. "Politics in evaluation: Politically responsive evaluation in high stakes environments," Evaluation and Program Planning, Elsevier, vol. 53(C), pages 44-56.
    4. Thayer, Colette E. & Fine, Allison H., 2001. "Evaluation and outcome measurement in the non-profit sector: stakeholder participation," Evaluation and Program Planning, Elsevier, vol. 24(1), pages 103-108, February.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jacobson, Miriam R. & Azzam, Tarek, 2018. "The effects of stakeholder involvement on perceptions of an evaluation’s credibility," Evaluation and Program Planning, Elsevier, vol. 68(C), pages 64-73.
    2. Ivehammar, Pernilla, 2014. "Valuing environmental quality in actual travel time savings – The Haningeleden road project in Stockholm," Research in Transportation Economics, Elsevier, vol. 48(C), pages 349-356.
    3. Eva Wanek & Sacha Bourgeois-Gironde & Alda Mari, 2024. "Desire, moral evaluation or sense of duty: The modal framing of stated preference elicitation," Environmental Values, , vol. 33(4), pages 434-459, August.
    4. Massey, Oliver T., 2011. "A proposed model for the analysis and interpretation of focus groups in evaluation research," Evaluation and Program Planning, Elsevier, vol. 34(1), pages 21-28, February.
    5. Tonin, Stefania, 2018. "Citizens’ perspectives on marine protected areas as a governance strategy to effectively preserve marine ecosystem services and biodiversity," Ecosystem Services, Elsevier, vol. 34(PB), pages 189-200.
    6. Richard T. Carson & Miko_aj Czajkowski, 2014. "The discrete choice experiment approach to environmental contingent valuation," Chapters, in: Stephane Hess & Andrew Daly (ed.), Handbook of Choice Modelling, chapter 9, pages 202-235, Edward Elgar Publishing.
    7. Moeltner, Klaus, 2019. "Bayesian nonlinear meta regression for benefit transfer," Journal of Environmental Economics and Management, Elsevier, vol. 93(C), pages 44-62.
    8. Drichoutis, Andreas C. & Vassilopoulos, Achilleas & Lusk, Jayson L. & Nayga, Rodolfo M. Jr., 2015. "Reference dependence, consequentiality and social desirability in value elicitation: A study of fair labor labeling," 143rd Joint EAAE/AAEA Seminar, March 25-27, 2015, Naples, Italy 202705, European Association of Agricultural Economists.
    9. Bernadeta Gołębiowska & Anna Bartczak & Mikołaj Czajkowski, 2020. "Energy Demand Management and Social Norms," Energies, MDPI, vol. 13(15), pages 1-20, July.
    10. Blomquist, Glenn C. & Coomes, Paul A. & Jepsen, Christopher & Koford, Brandon C. & Troske, Kenneth R., 2014. "Estimating the social value of higher education: willingness to pay for community and technical colleges," Journal of Benefit-Cost Analysis, Cambridge University Press, vol. 5(1), pages 3-41, January.
    11. John C. Whitehead, 2024. "Review of Contingent Valuation of Environmental Goods: A Comprehensive Critique. Edited by Daniel McFadden and Kenneth Train (2017): An Update," Working Papers 24-17, Department of Economics, Appalachian State University.
    12. Don Kenkel & Alan Mathios & Grace Phillips & Revathy Suryanarayana & Hua Wang & Sen Zeng, 2025. "Understanding the Demand‐Side of an Illegal Market: A Case Study of the Prohibition of Menthol Cigarettes," Health Economics, John Wiley & Sons, Ltd., vol. 34(5), pages 956-971, May.
    13. Lee, Kyung-Sook & Kim, Ju-Hee & Yoo, Seung-Hoon, 2021. "Would people pay a price premium for electricity from domestic wind power facilities? The case of South Korea," Energy Policy, Elsevier, vol. 156(C).
    14. Drichoutis, Andreas C. & Lusk, Jayson L. & Pappa, Valentina, 2016. "Elicitation formats and the WTA/WTP gap: A study of climate neutral foods," Food Policy, Elsevier, vol. 61(C), pages 141-155.
    15. Khanal, Puskar N. & Grebner, Donald L. & Munn, Ian A. & Grado, Stephen C. & Grala, Robert K. & Henderson, James E., 2017. "Evaluating non-industrial private forest landowner willingness to manage for forest carbon sequestration in the southern United States," Forest Policy and Economics, Elsevier, vol. 75(C), pages 112-119.
    16. Sebastian Blesse & Philipp Lergetporer & Justus Nover & Katharina Werner, 2023. "Transparency and Policy Competition: Experimental Evidence from German Citizens and Politicians," Rationality and Competition Discussion Paper Series 387, CRC TRR 190 Rationality and Competition.
    17. Abbie A. Rogers & Fiona L. Dempster & Jacob I. Hawkins & Robert J. Johnston & Peter C. Boxall & John Rolfe & Marit E. Kragt & Michael P. Burton & David J. Pannell, 2019. "Valuing non-market economic impacts from natural hazards," Natural Hazards: Journal of the International Society for the Prevention and Mitigation of Natural Hazards, Springer;International Society for the Prevention and Mitigation of Natural Hazards, vol. 99(2), pages 1131-1161, November.
    18. Moukam, Claudiane Yanick & Atewamba, Calvin, 2023. "Incorporating expert knowledge in the estimate of farmers’ opportunity cost of supplying environmental services in rural Cameroon," Bio-based and Applied Economics Journal, Italian Association of Agricultural and Applied Economics (AIEAA), vol. 12(3), October.
    19. Desvousges, William H. & Gard, Nicholas & Michael, Holly J. & Chance, Anne D., 2018. "Habitat and Resource Equivalency Analysis: A Critical Assessment," Ecological Economics, Elsevier, vol. 143(C), pages 74-89.

    More about this item

    Keywords

    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:evarev:v:40:y:2016:i:1:p:29-60. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.