IDEAS home Printed from https://ideas.repec.org/a/inm/ormsom/v25y2023i3p1051-1065.html

Detecting Human Trafficking: Automated Classification of Online Customer Reviews of Massage Businesses

Author

Listed:
  • Ruoting Li

    (Edward P. Fitts Department of Industrial and Systems Engineering, North Carolina State University, Raleigh, North Carolina 27695)

  • Margaret Tobey

    (Operations Research Graduate Program, North Carolina State University, Raleigh, North Carolina 27695)

  • Maria E. Mayorga

    (Edward P. Fitts Department of Industrial and Systems Engineering, North Carolina State University, Raleigh, North Carolina 27695)

  • Sherrie Caltagirone

    (Global Emancipation Network, Clermont, Florida 34715)

  • Osman Y. Özaltın

    (Edward P. Fitts Department of Industrial and Systems Engineering, North Carolina State University, Raleigh, North Carolina 27695)

Abstract

Problem definition : Approximately 11,000 alleged illicit massage businesses (IMBs) exist across the United States hidden in plain sight among legitimate businesses. These illicit businesses frequently exploit workers, many of whom are victims of human trafficking, forced or coerced to provide commercial sex. Academic/practical relevance: Although IMB review boards like Rubmaps.ch can provide first-hand information to identify IMBs, these sites are likely to be closed by law enforcement. Open websites like Yelp.com provide more accessible and detailed information about a larger set of massage businesses. Reviews from these sites can be screened for risk factors of trafficking. Methodology : We develop a natural language processing approach to detect online customer reviews that indicate a massage business is likely engaged in human trafficking. We label data sets of Yelp reviews using knowledge of known IMBs. We develop a lexicon of key words/phrases related to human trafficking and commercial sex acts. We then build two classification models based on this lexicon. We also train two classification models using embeddings from the bidirectional encoder representations from transformers (BERT) model and the Doc2Vec model. Results: We evaluate the performance of these classification models and various ensemble models. The lexicon-based models achieve high precision, whereas the embedding-based models have relatively high recall. The ensemble models provide a compromise and achieve the best performance on the out-of-sample test. Our results verify the usefulness of ensemble methods for building robust models to detect risk factors of human trafficking in reviews on open websites like Yelp. Managerial implications : The proposed models can save countless hours in IMB investigations by automatically sorting through large quantities of data to flag potential illicit activity, eliminating the need for manual screening of these reviews by law enforcement and other stakeholders.

Suggested Citation

  • Ruoting Li & Margaret Tobey & Maria E. Mayorga & Sherrie Caltagirone & Osman Y. Özaltın, 2023. "Detecting Human Trafficking: Automated Classification of Online Customer Reviews of Massage Businesses," Manufacturing & Service Operations Management, INFORMS, vol. 25(3), pages 1051-1065, May.
    • Handle: RePEc:inm:ormsom:v:25:y:2023:i:3:p:1051-1065
    DOI: 10.1287/msom.2023.1196
    as

    Download full text from publisher

    File URL: http://dx.doi.org/10.1287/msom.2023.1196
    Download Restriction: no
    File URL: https://libkey.io/10.1287/msom.2023.1196?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Luedtke Alexander R. & van der Laan Mark J., 2016. "Super-Learning of an Optimal Dynamic Treatment Rule," The International Journal of Biostatistics, De Gruyter, vol. 12(1), pages 305-332, May.
    2. Burcu B. Keskin & Gregory J. Bott & Nickolas K. Freeman, 2021. "Cracking Sex Trafficking: Data Analysis, Pattern Recognition, and Path Prediction," Production and Operations Management, Production and Operations Management Society, vol. 30(4), pages 1110-1135, April.
    3. Cheng Ju & Aurélien Bibaut & Mark van der Laan, 2018. "The relative performance of ensemble methods with deep convolutional neural networks for image classification," Journal of Applied Statistics, Taylor & Francis Journals, vol. 45(15), pages 2800-2818, November.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jin Wang & Donglin Zeng & D. Y. Lin, 2022. "Semiparametric single-index models for optimal treatment regimens with censored outcomes," Lifetime Data Analysis: An International Journal Devoted to Statistical Methods and Applications for Time-to-Event Data, Springer, vol. 28(4), pages 744-763, October.
    2. Hyungjin Ko & Jaewook Lee & Junyoung Byun & Bumho Son & Saerom Park, 2019. "Loss-Driven Adversarial Ensemble Deep Learning for On-Line Time Series Analysis," Sustainability, MDPI, vol. 11(12), pages 1-24, June.
    3. JoonBum Leem & Ha Young Kim, 2020. "Action-specialized expert ensemble trading system with extended discrete action space using deep reinforcement learning," PLOS ONE, Public Library of Science, vol. 15(7), pages 1-39, July.
    4. James Cussens & Julia Hatamyar & Vishalie Shah & Noemi Kreif, 2025. "Fast Learning of Optimal Policy Trees," Papers 2506.15435, arXiv.org.
    5. Nickolas K. Freeman & Burcu B. Keskin & Gregory J. Bott, 2022. "Collaborating with Local and Federal Law Enforcement for Disrupting Sex Trafficking Networks," Interfaces, INFORMS, vol. 52(5), pages 446-459, September.
    6. Doyun Kim & Joowon Chung & Jongmun Choi & Marc D. Succi & John Conklin & Maria Gabriela Figueiro Longo & Jeanne B. Ackman & Brent P. Little & Milena Petranovic & Mannudeep K. Kalra & Michael H. Lev & , 2022. "Accurate auto-labeling of chest X-ray images based on quantitative similarity to an explainable AI model," Nature Communications, Nature, vol. 13(1), pages 1-15, December.
    7. Mireia Crispin-Ortuzar & Ramona Woitek & Marika A. V. Reinius & Elizabeth Moore & Lucian Beer & Vlad Bura & Leonardo Rundo & Cathal McCague & Stephan Ursprung & Lorena Escudero Sanchez & Paula Martin-, 2023. "Integrated radiogenomics models predict response to neoadjuvant chemotherapy in high grade serous ovarian cancer," Nature Communications, Nature, vol. 14(1), pages 1-14, December.
    8. Pia Ramchandani & Hamsa Bastani & Emily Wyatt, 2025. "Unmasking Human Trafficking Risk in Commercial Sex Supply Chains with Machine Learning," Manufacturing & Service Operations Management, INFORMS, vol. 27(3), pages 700-719, May.
    9. Iftikhar Ahmad & Abdul Qayyum & Brij B. Gupta & Madini O. Alassafi & Rayed A. AlGhamdi, 2022. "Ensemble of 2D Residual Neural Networks Integrated with Atrous Spatial Pyramid Pooling Module for Myocardium Segmentation of Left Ventricle Cardiac MRI," Mathematics, MDPI, vol. 10(4), pages 1-23, February.
    10. Amine Belhadi & Sachin S. Kamble & Venkatesh Mani & Imane Benkhati & Fatima Ezahra Touriki, 2025. "An ensemble machine learning approach for forecasting credit risk of agricultural SMEs’ investments in agriculture 4.0 through supply chain finance," Annals of Operations Research, Springer, vol. 345(2), pages 779-807, February.
    11. I Díaz & O Savenkov & K Ballman, 2018. "Targeted learning ensembles for optimal individualized treatment rules with time-to-event outcomes," Biometrika, Biometrika Trust, vol. 105(3), pages 723-738.
    12. Qingliang Fan & Yu-Chin Hsu & Robert P. Lieli & Yichong Zhang, 2022. "Estimation of Conditional Average Treatment Effects With High-Dimensional Data," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 40(1), pages 313-327, January.
    13. Nickolas Freeman & Greg Bott & Burcu Keskin & Jason Parton & James Cochran, 2026. "Linking Multisite Sex Ad Data at the Individual Level to Aid Counter-Trafficking Efforts," Manufacturing & Service Operations Management, INFORMS, vol. 28(1), pages 133-152, January.
    14. Prakhar Bansal & Rahul Kumar & Somesh Kumar, 2021. "Disease Detection in Apple Leaves Using Deep Convolutional Neural Network," Agriculture, MDPI, vol. 11(7), pages 1-23, June.
    15. Keskin, Burcu B. & Griffin, Emily C. & Prell, Jonathan O. & Dilkina, Bistra & Ferber, Aaron & MacDonald, John & Hilend, Rowan & Griffis, Stanley & Gore, Meredith L., 2023. "Quantitative Investigation of Wildlife Trafficking Supply Chains: A Review," Omega, Elsevier, vol. 115(C).
    16. Xiaodan Xie & Felipe Aros‐Vera, 2022. "An interdependent network interdiction model for disrupting sex trafficking networks," Production and Operations Management, Production and Operations Management Society, vol. 31(6), pages 2695-2713, June.
    17. Enes Eryarsoy & Kazim Topuz & Cenk Demiroglu, 2024. "Disentangling human trafficking types and the identification of pathways to forced labor and sex: an explainable analytics approach," Annals of Operations Research, Springer, vol. 335(2), pages 761-795, April.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:ormsom:v:25:y:2023:i:3:p:1051-1065. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Asher (email available below). General contact details of provider: https://edirc.repec.org/data/inforea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.