IDEAS home Printed from https://ideas.repec.org/a/gam/jscscx/v9y2020i9p155-d410777.html
   My bibliography  Save this article

Does Scientific Evaluation Matter? Improving Digital Simulation Games by Design-Based Research

Author

Listed:
  • Sven Ivens

    (Department of Political Science, University of Goettingen, 37073 Goettingen, Germany)

  • Monika Oberle

    (Department of Political Science, University of Goettingen, 37073 Goettingen, Germany)

Abstract

Grounded in a design-based research approach, the aim of this article is to determine whether scientific evaluations help to (a) identify and fix problems in educational interventions and (b) eventually foster a more effective and positive evaluated intervention. Therefore, data from a longer-term evaluation of short digital simulation games about the European Parliament for civic education in schools were used. The data included three cycles of interventions with pre- and post-evaluations starting with the first prototype in 2015/2016 ( n = 209), the second cycle in 2017/18 ( n = 97), and the last one in 2019/20 ( n = 222). After each evaluation, major problems and critiques regarding the simulation game were discussed with the developers, and changes were implemented in the game design. The four most important problems, the processes by which they were improved and the reactions of the participants in the following evaluations are pointed out in the article. A comparison of the last and first evaluation cycle showed an overall improvement of the simulation game regarding its effectiveness in transferring EU knowledge and the participants’ general satisfaction with the simulation game. This study underlines the value of the design-based research approach for developing educational interventions and can be useful for further work on civic education measures and the implementation of digital simulation games.

Suggested Citation

  • Sven Ivens & Monika Oberle, 2020. "Does Scientific Evaluation Matter? Improving Digital Simulation Games by Design-Based Research," Social Sciences, MDPI, vol. 9(9), pages 1-14, September.
  • Handle: RePEc:gam:jscscx:v:9:y:2020:i:9:p:155-:d:410777
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2076-0760/9/9/155/pdf
    Download Restriction: no

    File URL: https://www.mdpi.com/2076-0760/9/9/155/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Anthony Onwuegbuzie & Nancy Leech, 2007. "Validity and Qualitative Research: An Oxymoron?," Quality & Quantity: International Journal of Methodology, Springer, vol. 41(2), pages 233-249, April.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Liyuan Liu & Steven Donbavand & Bryony Hoskins & Jan Germen Janmaat & Dimokritos Kavadias, 2021. "Measuring and Evaluating the Effectiveness of Active Citizenship Education Programmes to Support Disadvantaged Youth," Social Sciences, MDPI, vol. 10(10), pages 1-10, October.
    2. Sven Ivens & Gerlinde Wiese & Klaus Dittert & Oliver Mußhoff & Monika Oberle, 2020. "Bringing Policy Decisions to the People—Education for Sustainable Development through a Digital Simulation Game," Sustainability, MDPI, vol. 12(20), pages 1-18, October.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Arif Mostafa Khan & Meine Pieter van Dijk, 2024. "The role of multi‐stakeholder initiatives in advancing circularity and social sustainability in the textiles sector of Bangladesh," Journal of International Development, John Wiley & Sons, Ltd., vol. 36(3), pages 1765-1788, April.
    2. Seyyed Ostovar-Namaghi & Maryam Gholami, 2018. "Exploring Language Teachers’ Perceptions of Cyclical Presentation of Materials in an EFL Context: A Grounded Theory," English Language Teaching, Canadian Center of Science and Education, vol. 11(3), pages 1-60, March.
    3. Dietlmeier, Simon Frederic & Floetgen, Rob Jago & Urmetzer, Florian, 2024. "Performance of B2B Platform Partnership Management," MPRA Paper 120610, University Library of Munich, Germany.
    4. Hassan Raza, 2018. "Participatory action research: working beyond disaster toward prevention," Natural Hazards: Journal of the International Society for the Prevention and Mitigation of Natural Hazards, Springer;International Society for the Prevention and Mitigation of Natural Hazards, vol. 91(1), pages 117-131, March.
    5. Lukka, Kari & Modell, Sven, 2010. "Validation in interpretive management accounting research," Accounting, Organizations and Society, Elsevier, vol. 35(4), pages 462-477, May.
    6. Caspar, Sienna & Phinney, Alison & Spenceley, Shannon & Ratner, Pam, 2020. "Creating cultures of care: exploring the social organization of care delivery in long-term care homes," LSE Research Online Documents on Economics 106220, London School of Economics and Political Science, LSE Library.
    7. Rosario Vázquez-Carrasco & Ma. López-Pérez & Edgar Centeno, 2012. "A qualitative approach to the challenges for women in management: are they really starting in the 21st century?," Quality & Quantity: International Journal of Methodology, Springer, vol. 46(5), pages 1337-1357, August.
    8. Hamdan A. Alamri & Abdulaziz A. Alfayez, 2023. "Preservice teachers’ experiences of observing their teaching competencies via self-recorded videos in a personalized learning environment," Palgrave Communications, Palgrave Macmillan, vol. 10(1), pages 1-12, December.
    9. Karen Hidden & Jonathan Tresman Marks, 2020. "Misaligned Needs in the Pursuit of Shared Value: A Multi-Stakeholder Study of the Shift from Corporate Social Responsibility to Corporate Social Entrepreneurship in an Emerging Economy," Journal of Entrepreneurship and Innovation in Emerging Economies, Entrepreneurship Development Institute of India, vol. 6(2), pages 363-382, July.
    10. Joel Oommen George & Suzanne Elayan & Martin Sykora & Marin Solter & Rob Feick & Christopher Hewitt & Yiqiao Liu & Ketan Shankardass, 2023. "The Role of Social Media in Building Pandemic Resilience in an Urban Community: A Qualitative Case Study," IJERPH, MDPI, vol. 20(17), pages 1-18, September.
    11. John Robert Warren & Andrew Halpern-Manners, 2012. "Panel Conditioning in Longitudinal Social Science Surveys," Sociological Methods & Research, , vol. 41(4), pages 491-534, November.
    12. Milanova, Veselina & Maas, Peter, 2017. "Sharing intangibles: Uncovering individual motives for engagement in a sharing service setting," Journal of Business Research, Elsevier, vol. 75(C), pages 159-171.
    13. C. Poortman & K. Schildkamp, 2012. "Alternative quality standards in qualitative research?," Quality & Quantity: International Journal of Methodology, Springer, vol. 46(6), pages 1727-1751, October.
    14. Genevé Richard & Elza Odendaal, 2021. "Credibility-enhancing mechanisms, other than external assurance, in integrated reporting," Journal of Management & Governance, Springer;Accademia Italiana di Economia Aziendale (AIDEA), vol. 25(1), pages 61-93, March.
    15. Laumann, Karin, 2020. "Criteria for qualitative methods in human reliability analysis," Reliability Engineering and System Safety, Elsevier, vol. 194(C).
    16. Julia K. Groote & Sabrina Schell & Nadine Kammerlander & Andreas Hack, 2023. "The role of similarity and complementarity in the selection of potential partners for open innovation projects in family firms," Small Business Economics, Springer, vol. 60(4), pages 1347-1367, April.
    17. Sindre A. Høyland & Janne M. Hagen & Wolf Engelbach, 2017. "Developing and Applying a Framework for Assessing the Research Quality of Qualitative Project Methods in the EU Project SECUR-ED," SAGE Open, , vol. 7(2), pages 21582440177, May.
    18. Eeva‐Mari Ihantola & Lili‐Anne Kihn, 2011. "Threats to validity and reliability in mixed methods accounting research," Qualitative Research in Accounting & Management, Emerald Group Publishing Limited, vol. 8(1), pages 39-58, April.
    19. Santos, Paula Luisa Costa Teixeira & Monteiro, Paulo Adelino Antunes & Studic, Milena & Majumdar, Arnab, 2017. "A methodology used for the development of an Air Traffic Management functional system architecture," Reliability Engineering and System Safety, Elsevier, vol. 165(C), pages 445-457.
    20. Matthew M. Mars, 2022. "Community and Cultural Entrepreneurship and Value Co-Creation in the Local Food Marketscape," Sustainability, MDPI, vol. 14(24), pages 1-20, December.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jscscx:v:9:y:2020:i:9:p:155-:d:410777. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.