Automatic Information Extraction from the Web: An HMM-Based Approach

With the continued growth of the Internet and a huge amount of available data, extracting meaningful information from the Web has got a wide interest in both research community and business organizations. Although there exists a number of previous research works, to the best of our knowledge, none of them is flexible enough to fulfill users’ requirements in a variety of application domains. In this paper, we discuss and propose a general, extensible and dynamic approach based on the Hidden Markov model (HMM) in order to facilitate the efficient information extraction from HTML pages. Our proposed approach helps experts build a HMM from necessary specifications, train the system search engine, and extract meaningful information from HTML pages with the high precision and at a reasonable cost. More importantly, the proposed approach can be employed to support building knowledge bases for the next generation of the Web applications, i.e. the semantic Web. We developed and evaluated this model on a prototype, called PriceSearch, to extract price information of goods such as Nokia mobiles, computer mice, digital cameras. Experimental results confirm the efficiency of our theoretical analyses and approach.