Learning context-dependent word embeddings based on dependency parsing

Word embeddings constitute the basic methods of text representation. Whether they are the inputs to a machine learning algorithm or the features used in a natural language processing application, embeddings have proven helpful in solving various text processing tasks. In natural language texts, contextual information exerts a crucial influence on the semantics of word representations. In current research, most training models are based on shallow textual information and do not fully exploit deep relationships in sentences. To overcome this problem, this paper proposes the dependency-based continuous bag-of-words model which integrates the dependency relationships between words and sentences into the context with weights, thereby increasing the influence of specific contextual information on the prediction of target words. This method increases the abundancy of word context information and enhances the semantics of word embeddings. The experimental results show that the proposed method highlights semantic relations and improves the performance of word representations.