HW-STALKER: A machine learning-based system for transforming QURE-Pagelets to XML

In this paper, we address the problem of extracting and transforming dynamically generated hyperlinked hidden web query results to XML. Our approach is based on the STALKER approach. As STALKER was designed to extract data from a single web page, it cannot handle a set of hyperlinked pages. We propose an algorithm called HW-Transform for transforming hidden web query results (also called QURE-Pagelets) to XML format using machine learning by extending STALKER to handle hyperlinked hidden web pages. One of the key features of our approach is that we identify and transform key attributes of query results into XML attributes. These key attributes facilitate applications such as change detection and data integration by efficiently identifying related or identical results. Based on the proposed algorithm, we have implemented a prototype system called HW-STALKER using Java. Our experiments demonstrate that HW-Transform shows acceptable performance for transforming QURE-Pagelets to XML. (c) 2005 Elsevier B.V. All rights reserved.