Initialism disambiguation: Man versus machine

Disambiguation of ambiguous initialisms and acronyms is critical to the proper understanding of various types of texts. A model that attempts to solve this has previously been presented. This model contained various baseline features, including contextual relationship features, statistical features, and language‐specific features. The domain of Jewish law documents written in Hebrew and Aramaic is known to be rich in ambiguous abbreviations and therefore this model was implemented and applied over 2 separate corpuses within this domain. Several common machine‐learning (ML) methods were tested with the intent of finding a successful integration of the baseline feature variants. When the features were evaluated individually, the best averaged results were achieved by a library for support vector machines (LIBSVM); 98.07% of the ambiguous abbreviations, which were researched in the domain, were disambiguated correctly. When all the features were evaluated together, the J48 ML method achieved the best result, with 96.95% accuracy. In this paper, we examine the system's degree of success and the degree of its professionalism by conducting a comparison between this system's results and the results achieved by 39 participants, highly fluent in the research domain. Despite the fact that all the participants had backgrounds in religious scriptures and continue to study these texts, the system's accuracy rate, 98.07%, was significantly higher than the average accuracy result of the participants, 91.65%. Further analysis of the results for each corpus implies that participants overcomplicate the required task, as well as exclude vital information needed to properly examine the context of a given initialism.