IDEAS home Printed from https://ideas.repec.org/a/eee/epplan/v98y2023ics0149718923000514.html
   My bibliography  Save this article

Consequences of survey modification in a program evaluation: An exploratory research on evaluation study

Author

Listed:
  • Braman, Colin
  • Azzam, Tarek

Abstract

The purpose of this exploratory research on evaluation study was to examine how modified and unmodified scales of critical thinking and interest in science careers would affect the evaluation conclusions. Surveys measuring various outcomes that are used in program evaluation are frequently modified from their original versions in response to the unique context of programs. Modifying existing published surveys by removing or adding items can affect the psychometric properties of the original scale and may produce differing results. The results of the comparisons found that unmodified and modified surveys had similar reliabilities; however, one of the scales produced contradictory evaluation findings. Lessons learned from this study suggest that scales can be modified in evaluation, but great care is needed to address the potential strengths and limitations of the modified scale and balance the technical needs with responsiveness to program context.

Suggested Citation

  • Braman, Colin & Azzam, Tarek, 2023. "Consequences of survey modification in a program evaluation: An exploratory research on evaluation study," Evaluation and Program Planning, Elsevier, vol. 98(C).
    • Handle: RePEc:eee:epplan:v:98:y:2023:i:c:s0149718923000514
    DOI: 10.1016/j.evalprogplan.2023.102274
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0149718923000514
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.evalprogplan.2023.102274?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Greene, Jennifer C., 1990. "Technical quality versus user responsiveness in evaluation practice," Evaluation and Program Planning, Elsevier, vol. 13(3), pages 267-274, January.
    2. Flynn, Leisa Reinecke & Goldsmith, Ronald E., 1999. "A Short, Reliable Measure of Subjective Knowledge," Journal of Business Research, Elsevier, vol. 46(1), pages 57-66, September.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Groza, Mark D. & Groza, Mya Pronschinske, 2018. "Salesperson regulatory knowledge and sales performance," Journal of Business Research, Elsevier, vol. 89(C), pages 37-46.
    2. Catherine Viot & Juliette Passebois-Ducros, 2010. "Wine brands or branded wines? The specificity of the French market in terms of the brand," Post-Print hal-01803728, HAL.
    3. Eva M. Murgado-Armenteros & María Gutierrez-Salcedo & Francisco José Torres-Ruiz, 2020. "The Concern about Biodiversity as a Criterion for the Classification of the Sustainable Consumer: A Cross-Cultural Approach," Sustainability, MDPI, vol. 12(8), pages 1-14, April.
    4. Kwon, Kyoung-Nan & Lee, Jinkook, 2009. "The effects of reference point, knowledge, and risk propensity on the evaluation of financial products," Journal of Business Research, Elsevier, vol. 62(7), pages 719-725, July.
    5. Casenave, Eric & Klarmann, Martin, 2020. "The accountability paradox: How holding marketers accountable hinders alignment with short-term marketing goals," Journal of Business Research, Elsevier, vol. 112(C), pages 95-108.
    6. Christian Hildebrand & Gerald Häubl & Andreas Herrmann & Jan R. Landwehr, 2013. "When Social Media Can Be Bad for You: Community Feedback Stifles Consumer Creativity and Reduces Satisfaction with Self-Designed Products," Information Systems Research, INFORMS, vol. 24(1), pages 14-29, March.
    7. Wei, Wei & Kim, Gaeul & Miao, Li & Behnke, Carl & Almanza, Barbara, 2018. "Consumer inferences of corporate social responsibility (CSR) claims on packaged foods," Journal of Business Research, Elsevier, vol. 83(C), pages 186-201.
    8. Aksoy, Lerzan & Cooil, Bruce & Lurie, Nicholas H., 2011. "Decision Quality Measures in Recommendation Agents Research," Journal of Interactive Marketing, Elsevier, vol. 25(2), pages 110-122.
    9. Rai, Dipankar & Lin, Chien-Wei (Wilson), 2019. "The influence of implicit self-theories on consumer financial decision making," Journal of Business Research, Elsevier, vol. 95(C), pages 316-325.
    10. Yajuan Dong & Yi Jin & Khan Anwar Kamal, 2023. "The Impact of Perceived Quality of Government Subsidies on Residents’ Travel Intention Post-COVID-19 in Wuhan, China," Sustainability, MDPI, vol. 15(14), pages 1-18, July.
    11. Lin, Zhibin & Filieri, Raffaele, 2015. "Airline passengers’ continuance intention towards online check-in services: The role of personal innovativeness and subjective knowledge," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 81(C), pages 158-168.
    12. Luis Fernando Hor-Meyll & Marcel Correia de Lima & Jorge Brantes Ferreira, 2014. "Why Should I Accept Ads on my Mobile Phone? Factors Affecting the Acceptance by Brazilian Teenagers," Brazilian Business Review, Fucape Business School, vol. 11(4), pages 130-150, August.
    13. Julian Löbbers & Sebastian Lins & Theresa Kromat & Alexander Benlian & Ali Sunyaev, 2022. "A multi-perspective lens on web assurance seals: contrasting vendors’ intended and consumers’ perceived effects," Electronic Commerce Research, Springer, vol. 22(4), pages 1573-1615, December.
    14. Krittinee Nuttavuthisit & John Thøgersen, 2017. "The Importance of Consumer Trust for the Emergence of a Market for Green Products: The Case of Organic Food," Journal of Business Ethics, Springer, vol. 140(2), pages 323-337, January.
    15. Schweisfurth, Tim G. & Raasch, Christina, 2018. "Absorptive capacity for need knowledge: Antecedents and effects for employee innovativeness," Research Policy, Elsevier, vol. 47(4), pages 687-699.
    16. Dutta, Sujay & Banerjee, Somak & Johnson, Aaron & Biswas, Abhijit, 2022. "Overcoming the challenge of low familiarity: Can a weakly familiar brand signal quality with exceptionally strong warranty?," Journal of Business Research, Elsevier, vol. 141(C), pages 737-754.
    17. Schwartz, Robert & Mayne, John, 2005. "Assuring the quality of evaluative information: theory and practice," Evaluation and Program Planning, Elsevier, vol. 28(1), pages 1-14.
    18. van Rijnsoever, Frank J. & Farla, Jacco C.M., 2014. "Identifying and explaining public preferences for the attributes of energy technologies," Renewable and Sustainable Energy Reviews, Elsevier, vol. 31(C), pages 71-82.
    19. Andreas Herrmann & René Befurt & Mark Heitmann & Hans Berger, 2007. "Alles für die Marke? Produktdesign im Konflikt zwischen einer markenkonformen und einer eigenständigen Produktliniengestaltung," Schmalenbach Journal of Business Research, Springer, vol. 59(8), pages 1055-1079, December.
    20. L. Ende & M.-A. Reinhard & L. Göritz, 2023. "Detecting Greenwashing! The Influence of Product Colour and Product Price on Consumers’ Detection Accuracy of Faked Bio-fashion," Journal of Consumer Policy, Springer, vol. 46(2), pages 155-189, June.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:epplan:v:98:y:2023:i:c:s0149718923000514. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/evalprogplan .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.