Data-Driven Approaches to Selecting Samples for Training Neural Networks

Modern neural networks, that are now commonly used for most natural language processing (NLP) tasks, contain many hidden units and parameters. There is a considerable interest in developing strategies for selecting an optimal set of samples to train such large models for biomedical tasks because developing training data is expensive and time consuming in the biomedical space. Lack of sufficient training data is exacerbated by the fact that the ratio of negative samples to positive samples is also highly skewed, i.e., too many negative samples but too few positive samples. Therefore, an important problem, especially for the biomedical space, what is the optimum set of negative samples to use in creating an effective and balanced training data sample. Interestingly though, the insights which may help to decide the most effective sample selection can be found in the data itself (i.e., in the samples themselves). This chapter briefly reviews traditional approaches to selecting training samples and then presents the latest data-driven approaches for selecting samples to effectively train modern neural networks.