IDEAS home Printed from https://ideas.repec.org/a/plo/pone00/0211884.html

Situational Judgment Tests as a method for measuring personality: Development and validity evidence for a test of Dependability

Author

Listed:
  • Gabriel Olaru
  • Jeremy Burrus
  • Carolyn MacCann
  • Franklin M Zaromb
  • Oliver Wilhelm
  • Richard D Roberts

Abstract

Situational Judgment Tests (SJTs) are criterion valid low fidelity measures that have gained much popularity as predictors of job performance. A broad variety of SJTs have been studied, but SJTs measuring personality are still rare. Personality traits such as Conscientiousness are valid predictors of many educational, work and life-related outcomes and SJTs are less prone to faking than classical self-report measurements. We developed an SJT measure of Dependability, a core facet of Conscientiousness, by gathering critical incidents in semi-structured interviews using the construct definition of Dependability as a prompt. We examined the psychometric properties of the newly developed SJTs across two studies (N = 546 general population; N = 440 sales professionals). The internal validity of the SJTs was examined by correlating the SJT scores with related self-report measures of Dependability and Conscientiousness, as well as testing the unidimensionality of the measure with CFA. Additionally, we specified a bi-factor model of SJT, self-report and behavioral checklist measures of Dependability accounting for common and specific measurement variance. External validity was examined by correlating the SJT scale and specific factor with work-related outcomes. The results show that the Dependability SJTs with an expert based scoring procedure were psychometrically sound and correlated moderately to highly with traditional self-report measures of Dependability and Conscientiousness. However, a large proportion of SJT variance cannot be accounted for by personality alone. This supports the notion that SJTs measure general domain knowledge about the effectiveness of personality-related behaviors. We conclude that SJT measures of personality can be a promising addition to classical self-report assessments and can be used in a wide variety of applications beyond measurement and selection, for instance as formative assessments of personality.

Suggested Citation

  • Gabriel Olaru & Jeremy Burrus & Carolyn MacCann & Franklin M Zaromb & Oliver Wilhelm & Richard D Roberts, 2019. "Situational Judgment Tests as a method for measuring personality: Development and validity evidence for a test of Dependability," PLOS ONE, Public Library of Science, vol. 14(2), pages 1-19, February.
    • Handle: RePEc:plo:pone00:0211884
    DOI: 10.1371/journal.pone.0211884
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0211884
    Download Restriction: no
    File URL: https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0211884&type=printable
    Download Restriction: no
    File URL: https://libkey.io/10.1371/journal.pone.0211884?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Gabriele Paolacci & Jesse Chandler & Panagiotis G. Ipeirotis, 2010. "Running experiments on Amazon Mechanical Turk," Judgment and Decision Making, Society for Judgment and Decision Making, vol. 5(5), pages 411-419, August.
    2. Denny Borsboom, 2006. "The attack of the psychometricians," Psychometrika, Springer;The Psychometric Society, vol. 71(3), pages 425-440, September.
    3. Lievens, Filip & Motowidlo, Stephan J., 2016. "Situational Judgment Tests: From Measures of Situational Judgment to Measures of General Domain Knowledge," Industrial and Organizational Psychology, Cambridge University Press, vol. 9(1), pages 3-22, March.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Koller, Sonia & Ahmetoglu, Gorkan & Stephan, Ute, 2025. "Measuring entrepreneurs' use of effectuation as heuristics: Development and validation of a situational judgment test (SJT) for effectuation," Journal of Business Venturing, Elsevier, vol. 40(6).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Yamada, Katsunori & Sato, Masayuki, 2013. "Another avenue for anatomy of income comparisons: Evidence from hypothetical choice experiments," Journal of Economic Behavior & Organization, Elsevier, vol. 89(C), pages 35-57.
    2. Kenneth D. Nguyen & Heather Rosoff & Richard S. John, 2017. "Valuing Equal Protection in Aviation Security Screening," Risk Analysis, John Wiley & Sons, vol. 37(12), pages 2405-2419, December.
    3. Steven Andrew Culpepper & Herman Aguinis & Justin L. Kern & Roger Millsap, 2019. "High-Stakes Testing Case Study: A Latent Variable Approach for Assessing Measurement and Prediction Invariance," Psychometrika, Springer;The Psychometric Society, vol. 84(1), pages 285-309, March.
    4. Varun Dutt & Cleotilde Gonzalez, 2013. "Enabling Eco-Friendly Choices by Relying on the Proportional-Thinking Heuristic," Sustainability, MDPI, vol. 5(1), pages 1-15, January.
    5. Sweldens, Steven & Puntoni, Stefano & Paolacci, Gabriele & Vissers, Maarten, 2014. "The bias in the bias: Comparative optimism as a function of event social undesirability," Organizational Behavior and Human Decision Processes, Elsevier, vol. 124(2), pages 229-244.
    6. S. Venus Jin & Aziz Muqaddam, 2019. "Product placement 2.0: “Do Brands Need Influencers, or Do Influencers Need Brands?”," Journal of Brand Management, Palgrave Macmillan, vol. 26(5), pages 522-537, September.
    7. Enrico Rubaltelli & Lorella Lotto & Ilana Ritov & Rino Rumiati, 2015. "Moral investing: Psychological motivations and implications," Judgment and Decision Making, Society for Judgment and Decision Making, vol. 10(1), pages 64-75, January.
    8. Hsu, Dan K. & Burmeister-Lamp, Katrin & Simmons, Sharon A. & Foo, Maw-Der & Hong, Michelle C. & Pipes, Jesse D., 2019. "“I know I can, but I don't fit”: Perceived fit, self-efficacy, and entrepreneurial intention," Journal of Business Venturing, Elsevier, vol. 34(2), pages 311-326.
    9. Lutz, Christoph & Newlands, Gemma, 2018. "Consumer segmentation within the sharing economy: The case of Airbnb," Journal of Business Research, Elsevier, vol. 88(C), pages 187-196.
    10. Nir Halevy & Sora Jun & Eileen Y. Chou, 2020. "Intergroup Conflict is Our Business: CEOs’ Ethical Intergroup Leadership Fuels Stakeholder Support for Corporate Intergroup Responsibility," Journal of Business Ethics, Springer, vol. 162(1), pages 229-246, February.
    11. Mariconda, Simone & Lurati, Francesco, 2015. "Does familiarity breed stability? The role of familiarity in moderating the effects of new information on reputation judgments," Journal of Business Research, Elsevier, vol. 68(5), pages 957-964.
    12. Tarcia Camily Cavalcante Quezado & Nuno Fortes & William Quezado Figueiredo Cavalcante, 2022. "The Influence of Corporate Social Responsibility and Business Ethics on Brand Fidelity: The Importance of Brand Love and Brand Attitude," Sustainability, MDPI, vol. 14(5), pages 1-20, March.
    13. Gandullia, Luca & Lezzi, Emanuela, 2018. "The price elasticity of charitable giving: New experimental evidence," Economics Letters, Elsevier, vol. 173(C), pages 88-91.
    14. Pam A. Mueller & Lawrence M. Solan & John M. Darley, 2012. "When Does Knowledge Become Intent? Perceiving the Minds of Wrongdoers," Journal of Empirical Legal Studies, John Wiley & Sons, vol. 9(4), pages 859-892, December.
    15. Masha Shunko & Julie Niederhoff & Yaroslav Rosokha, 2018. "Humans Are Not Machines: The Behavioral Impact of Queueing Design on Service Time," Management Science, INFORMS, vol. 64(1), pages 453-473, January.
    16. Tobias Schlager & Ashley V. Whillans, 2022. "People underestimate the probability of contracting the coronavirus from friends," Humanities and Social Sciences Communications, Palgrave Macmillan, vol. 9(1), pages 1-11, December.
    17. Giorgia Ponsi & Maria Serena Panasiti & Salvatore Maria Aglioti & Marco Tullio Liuzza, 2017. "Right-wing authoritarianism and stereotype-driven expectations interact in shaping intergroup trust in one-shot vs multiple-round social interactions," PLOS ONE, Public Library of Science, vol. 12(12), pages 1-23, December.
    18. Charness, Gary & Gneezy, Uri & Kuhn, Michael A., 2013. "Experimental methods: Extra-laboratory experiments-extending the reach of experimental economics," Journal of Economic Behavior & Organization, Elsevier, vol. 91(C), pages 93-100.
    19. Anthony M. Evans & Kyle D. Dillon & Gideon Goldin & Joachim I. Krueger, 2011. "Trust and self-control: The moderating role of the default," Judgment and Decision Making, Society for Judgment and Decision Making, vol. 6(7), pages 697-705, October.
    20. Gerhard, Patrick & Hoffmann, Arvid O.I. & Post, Thomas, 2017. "Past performance framing and investors’ belief updating: Is seeing long-term returns always associated with smaller belief updates?," Journal of Behavioral and Experimental Finance, Elsevier, vol. 15(C), pages 38-51.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pone00:0211884. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosone (email available below). General contact details of provider: https://journals.plos.org/plosone/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.