IDEAS home Printed from https://ideas.repec.org/a/sae/sagope/v8y2018i2p2158244018773143.html
   My bibliography  Save this article

Simulating the Fidelity of Data for Large Stimulus Set Sizes and Variable Dimension Estimation in Multidimensional Scaling

Author

Listed:
  • Michael C. Hout
  • Corbin A. Cunningham
  • Arryn Robbins
  • Justin MacDonald

Abstract

Multidimensional scaling (MDS) is a statistical technique commonly used to model the psychological similarity among sets of stimulus items. Typically, MDS has been used with relatively small stimulus sets (30 items or fewer), in part due to the laborious nature of computational analysis and data collection. Modern computing power and newly advanced techniques for speeding data collection have made it possible to conduct MDS with many more stimuli. However, it is as yet unclear if MDS is as well-equipped to model the similarity of large stimulus sets as it is for more modest ones. Here, we conducted 337,500 simulation experiments, wherein hypothetical “true†MDS spaces were created, along with error-perturbed data from simulated “participants.†We examined the fidelity with which the spaces resulting from our “participants†captured the organization of the “true†spaces, as a function of item set size, amount of error in the data (i.e., noise), and dimensionality estimation. We found that although higher set sizes decrease model fit (i.e., they produce increased “stress†), they largely tended to increase determinacy of MDS spaces. These results are predicated, however, on the appropriate estimation of dimensionality of the MDS space. We argue that it is not only reasonable to adopt large stimulus set sizes but tends to be advantageous to do so. Applying MDS to larger sets is appealing, as it affords researchers greater flexibility in stimulus selection, more opportunity for exploration of their stimuli, and a higher likelihood that observed relationships are not due to stimulus-specific idiosyncrasies.

Suggested Citation

  • Michael C. Hout & Corbin A. Cunningham & Arryn Robbins & Justin MacDonald, 2018. "Simulating the Fidelity of Data for Large Stimulus Set Sizes and Variable Dimension Estimation in Multidimensional Scaling," SAGE Open, , vol. 8(2), pages 21582440187, April.
  • Handle: RePEc:sae:sagope:v:8:y:2018:i:2:p:2158244018773143
    DOI: 10.1177/2158244018773143
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/2158244018773143
    Download Restriction: no

    File URL: https://libkey.io/10.1177/2158244018773143?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Harvey Cohen & Lawrence Jones, 1974. "The effects of random error and subsampling of dimensions on recovery of configurations by non-metric multidimensional scaling," Psychometrika, Springer;The Psychometric Society, vol. 39(1), pages 69-90, March.
    2. Paul Isaac & David Poor, 1974. "On the determination of appropriate dimensionality in data with error," Psychometrika, Springer;The Psychometric Society, vol. 39(1), pages 91-109, March.
    3. J. Kruskal, 1964. "Multidimensional scaling by optimizing goodness of fit to a nonmetric hypothesis," Psychometrika, Springer;The Psychometric Society, vol. 29(1), pages 1-27, March.
    4. Forrest Young, 1970. "Nonmetric multidimensional scaling: Recovery of metric information," Psychometrika, Springer;The Psychometric Society, vol. 35(4), pages 455-473, December.
    5. Charles Sherman, 1972. "Nonmetric multidimensional scaling: A monte carlo study of the basic parameters," Psychometrika, Springer;The Psychometric Society, vol. 37(3), pages 323-355, September.
    6. Ian Spence & Forrest Young, 1978. "Monte Carlo studies in nonmetric scaling," Psychometrika, Springer;The Psychometric Society, vol. 43(1), pages 115-117, March.
    7. Michael C Hout & Stephen D Goldinger & Kyle J Brady, 2014. "MM-MDS: A Multidimensional Scaling Database with Similarity Ratings for 240 Object Categories from the Massive Memory Picture Database," PLOS ONE, Public Library of Science, vol. 9(11), pages 1-11, November.
    8. Roger Shepard, 1957. "Stimulus and response generalization: A stochastic model relating generalization to distance in psychological space," Psychometrika, Springer;The Psychometric Society, vol. 22(4), pages 325-345, December.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ian Spence, 1972. "A monte carlo evaluation of three nonmetric multidimensional scaling algorithms," Psychometrika, Springer;The Psychometric Society, vol. 37(4), pages 461-486, December.
    2. Roger Shepard, 1974. "Representation of structure in similarity data: Problems and prospects," Psychometrika, Springer;The Psychometric Society, vol. 39(4), pages 373-421, December.
    3. Roger Girard & Norman Cliff, 1976. "A monte carlo evaluation of interactive multidimensional scaling," Psychometrika, Springer;The Psychometric Society, vol. 41(1), pages 43-64, March.
    4. Michael Brusco & Stephanie Stahl, 2001. "Compact integer-programming models for extracting subsets of stimuli from confusion matrices," Psychometrika, Springer;The Psychometric Society, vol. 66(3), pages 405-419, September.
    5. Jerzy Grobelny & Rafal Michalski & Gerhard-Wilhelm Weber, 2021. "Modeling human thinking about similarities by neuromatrices in the perspective of fuzzy logic," WORking papers in Management Science (WORMS) WORMS/21/09, Department of Operations Research and Business Intelligence, Wroclaw University of Science and Technology.
    6. Phipps Arabie, 1991. "Was euclid an unnecessarily sophisticated psychologist?," Psychometrika, Springer;The Psychometric Society, vol. 56(4), pages 567-587, December.
    7. George Rabinowitz, 1976. "A procedure for ordering object pairs consistent with the multidimensional unfolding model," Psychometrika, Springer;The Psychometric Society, vol. 41(3), pages 349-373, September.
    8. Robert MacCallum, 1979. "Recovery of structure in incomplete data by alscal," Psychometrika, Springer;The Psychometric Society, vol. 44(1), pages 69-74, March.
    9. Joseph Zinnes & David MacKay, 1983. "Probabilistic multidimensional scaling: Complete and incomplete data," Psychometrika, Springer;The Psychometric Society, vol. 48(1), pages 27-48, March.
    10. Henry Brady, 1985. "Statistical consistency and hypothesis testing for nonmetric multidimensional scaling," Psychometrika, Springer;The Psychometric Society, vol. 50(4), pages 509-537, December.
    11. Scott R. Rosas, 2017. "Multi-map comparison for group concept mapping: an approach for examining conceptual congruence through spatial correspondence," Quality & Quantity: International Journal of Methodology, Springer, vol. 51(6), pages 2421-2439, November.
    12. Forrest Young & Cynthia Null, 1978. "Multidimensional scaling of nominal data: The recovery of metric information with alscal," Psychometrika, Springer;The Psychometric Society, vol. 43(3), pages 367-379, September.
    13. Giovanna Boccuzzo & Licia Maron, 2017. "Proposal of a composite indicator of job quality based on a measure of weighted distances," Quality & Quantity: International Journal of Methodology, Springer, vol. 51(5), pages 2357-2374, September.
    14. Jong-Seok Lee & Dan Zhu, 2012. "Shilling Attack Detection---A New Approach for a Trustworthy Recommender System," INFORMS Journal on Computing, INFORMS, vol. 24(1), pages 117-131, February.
    15. Ma, Jie & Tse, Ying Kei & Wang, Xiaojun & Zhang, Minhao, 2019. "Examining customer perception and behaviour through social media research – An empirical study of the United Airlines overbooking crisis," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 127(C), pages 192-205.
    16. Muñoz-Mas, Rafael & Vezza, Paolo & Alcaraz-Hernández, Juan Diego & Martínez-Capel, Francisco, 2016. "Risk of invasion predicted with support vector machines: A case study on northern pike (Esox Lucius, L.) and bleak (Alburnus alburnus, L.)," Ecological Modelling, Elsevier, vol. 342(C), pages 123-134.
    17. Venera Tomaselli, 1996. "Multivariate statistical techniques and sociological research," Quality & Quantity: International Journal of Methodology, Springer, vol. 30(3), pages 253-276, August.
    18. Simensen, Trond & Halvorsen, Rune & Erikstad, Lars, 2018. "Methods for landscape characterisation and mapping: A systematic review," Land Use Policy, Elsevier, vol. 75(C), pages 557-569.
    19. Marie Diekmann & Ludwig Theuvsen, 2019. "Value structures determining community supported agriculture: insights from Germany," Agriculture and Human Values, Springer;The Agriculture, Food, & Human Values Society (AFHVS), vol. 36(4), pages 733-746, December.
    20. Bijmolt, T.H.A. & Wedel, M., 1996. "A Monte Carlo Evaluation of Maximum Likelihood Multidimensional Scaling Methods," Other publications TiSEM f72cc9d8-f370-43aa-a224-4, Tilburg University, School of Economics and Management.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:sagope:v:8:y:2018:i:2:p:2158244018773143. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: . General contact details of provider: .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.