IDEAS home Printed from https://ideas.repec.org/a/spr/infsem/v21y2023i3d10.1007_s10257-023-00641-3.html
   My bibliography  Save this article

VAST: a practical validation framework for e-assessment solutions

Author

Listed:
  • Max Haastrecht

    (Leiden University)

  • Matthieu Brinkhuis

    (Utrecht University)

  • Saskia Wools

    (Stichting Cito)

  • Marco Spruit

    (Leiden University
    Leiden University Medical Center (LUMC))

Abstract

The influx of technology in education has made it increasingly difficult to assess the validity of educational assessments. The field of information systems often ignores the social dimension during validation, whereas educational research neglects the technical dimensions of designed instruments. The inseparability of social and technical elements forms the bedrock of socio-technical systems. Therefore, the current lack of validation approaches that address both dimensions is a significant gap. We address this gap by introducing VAST: a validation framework for e-assessment solutions. Examples of such solutions are technology-enhanced learning systems and e-health applications. Using multi-grounded action research as our methodology, we investigate how we can synthesise existing knowledge from information systems and educational measurement to construct our validation framework. We develop an extensive user guideline complementing our framework and find through expert interviews that VAST facilitates a comprehensive, practical approach to validating e-assessment solutions.

Suggested Citation

  • Max Haastrecht & Matthieu Brinkhuis & Saskia Wools & Marco Spruit, 2023. "VAST: a practical validation framework for e-assessment solutions," Information Systems and e-Business Management, Springer, vol. 21(3), pages 603-627, September.
    • Handle: RePEc:spr:infsem:v:21:y:2023:i:3:d:10.1007_s10257-023-00641-3
    DOI: 10.1007/s10257-023-00641-3
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s10257-023-00641-3
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s10257-023-00641-3?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Göran Goldkuhl & Stefan Cronholm & Mikael Lind, 2020. "Multi-grounded action research," Information Systems and e-Business Management, Springer, vol. 18(2), pages 121-156, June.
    2. John Venable & Jan Pries-Heje & Richard Baskerville, 2016. "FEDS: a Framework for Evaluation in Design Science Research," European Journal of Information Systems, Taylor & Francis Journals, vol. 25(1), pages 77-89, January.
    3. David A. Broniatowski & Conrad Tucker, 2017. "Assessing causal claims about complex engineered systems with quantitative data: internal, external, and construct validity," Systems Engineering, John Wiley & Sons, vol. 20(6), pages 483-496, November.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Raffaele Fabio Ciriello & Alexandra Cecilie Gjøl Torbensen & Magnus Rotvit Perlt Hansen & Christoph Müller-Bloch, 2023. "Blockchain-based digital rights management systems: Design principles for the music industry," Electronic Markets, Springer;IIM University of St. Gallen, vol. 33(1), pages 1-21, December.
    2. Myriam Schaschek & Fabian Gwinner & Nicolas Neis & Christoph Tomitza & Christian Zeiß & Axel Winkelmann, 2024. "Managing next generation BP-x initiatives," Information Systems and e-Business Management, Springer, vol. 22(3), pages 457-500, September.
    3. Andreas Engelmann & Gerhard Schwabe, 2024. "Certified data chats for future used car markets," Electronic Markets, Springer;IIM University of St. Gallen, vol. 34(1), pages 1-22, December.
    4. Alexandra Hoess & Jonathan Lautenschlager & Johannes Sedlmeir & Gilbert Fridgen & Vincent Schlatt & Nils Urbach, 2025. "Toward Seamless Mobility-as-a-Service," Business & Information Systems Engineering: The International Journal of WIRTSCHAFTSINFORMATIK, Springer;Gesellschaft für Informatik e.V. (GI), vol. 67(2), pages 149-170, April.
    5. Paola Lara Machado & Montijn Ven & Banu Aysolmaz & Oktay Turetken & Jan Brocke, 2024. "Navigating Business Model Redesign: The Compass Method for Identifying Changes to the Operating Model," Business & Information Systems Engineering: The International Journal of WIRTSCHAFTSINFORMATIK, Springer;Gesellschaft für Informatik e.V. (GI), vol. 66(5), pages 607-638, October.
    6. Tom Lewandowski & Emir Kučević & Stephan Leible & Mathis Poser & Tilo Böhmann, 2023. "Enhancing conversational agents for successful operation: A multi-perspective evaluation approach for continuous improvement," Electronic Markets, Springer;IIM University of St. Gallen, vol. 33(1), pages 1-20, December.
    7. Hendrik Haße & Hendrik Valk & Frederik Möller & Boris Otto, 2022. "Design Principles for Shared Digital Twins in Distributed Systems," Business & Information Systems Engineering: The International Journal of WIRTSCHAFTSINFORMATIK, Springer;Gesellschaft für Informatik e.V. (GI), vol. 64(6), pages 751-772, December.
    8. Carolin Stein & Timm Teubner & Stefan Morana, 2024. "Designing a conversational agent for supporting data exploration in citizen science," Electronic Markets, Springer;IIM University of St. Gallen, vol. 34(1), pages 1-18, December.
    9. Christoph Stoiber & Maximilian Stöter & Ludwig Englbrecht & Stefan Schönig & Björn Häckel, 2023. "Keeping Your Maturity Assessment Alive," Business & Information Systems Engineering: The International Journal of WIRTSCHAFTSINFORMATIK, Springer;Gesellschaft für Informatik e.V. (GI), vol. 65(6), pages 703-721, December.
    10. Joshua Holstein & Max Schemmer & Johannes Jakubik & Michael Vössing & Gerhard Satzger, 2023. "Sanitizing data for analysis: Designing systems for data understanding," Electronic Markets, Springer;IIM University of St. Gallen, vol. 33(1), pages 1-18, December.
    11. Laleh Rafati & Geert Poels, 2017. "Value-Driven Strategic Sourcing Based on Service-Dominant Logic," Service Science, INFORMS, vol. 9(4), pages 275-287, December.
    12. Katharina Breiter & Carlotta Crome & Anna Maria Oberländer & Feline Schnaak, 2024. "Dynamic Capabilities for the Twin Transformation Climb: A Capability Maturity Model," Information Systems Frontiers, Springer, vol. 26(6), pages 2205-2226, December.
    13. Moutinho, José da Assunção & Fernandes, Gabriela & Rabechini, Roque, 2024. "Evaluation in design science: A framework to support project studies in the context of University Research Centres," Evaluation and Program Planning, Elsevier, vol. 102(C).
    14. Julius Peter Landwehr & Niklas Kühl & Jannis Walk & Mario Gnädig, 2022. "Design Knowledge for Deep-Learning-Enabled Image-Based Decision Support Systems," Business & Information Systems Engineering: The International Journal of WIRTSCHAFTSINFORMATIK, Springer;Gesellschaft für Informatik e.V. (GI), vol. 64(6), pages 707-728, December.
    15. Christian Meske & Enrico Bunde, 2023. "Design Principles for User Interfaces in AI-Based Decision Support Systems: The Case of Explainable Hate Speech Detection," Information Systems Frontiers, Springer, vol. 25(2), pages 743-773, April.
    16. Ali Sunyaev & Tobias Dehling & Susanne Strahringer & Li Xu & Martin Heinig & Michael Perscheid & Rainer Alt & Matti Rossi, 2023. "The Future of Enterprise Information Systems," Business & Information Systems Engineering: The International Journal of WIRTSCHAFTSINFORMATIK, Springer;Gesellschaft für Informatik e.V. (GI), vol. 65(6), pages 731-751, December.
    17. Fabian Sven Karst & Mahei Manhai Li & Jan Marco Leimeister, 2025. "SynDEc: A Synthetic Data Ecosystem," Electronic Markets, Springer;IIM University of St. Gallen, vol. 35(1), pages 1-28, December.
    18. João Barata & Paulo Rupino Cunha & António Dias Figueiredo, 2023. "Self-reporting Limitations in Information Systems Design Science Research," Business & Information Systems Engineering: The International Journal of WIRTSCHAFTSINFORMATIK, Springer;Gesellschaft für Informatik e.V. (GI), vol. 65(2), pages 143-160, April.
    19. Hund, Lauren & Schroeder, Benjamin, 2020. "A causal perspective on reliability assessment," Reliability Engineering and System Safety, Elsevier, vol. 195(C).
    20. Dedi I. Inan & Ghassan Beydoun & Biswajeet Pradhan, 2022. "Disaster Management Knowledge Analysis Framework Validated," Information Systems Frontiers, Springer, vol. 24(6), pages 2077-2097, December.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:infsem:v:21:y:2023:i:3:d:10.1007_s10257-023-00641-3. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.