Evolutionary approach for semantic-based query sampling in large-scale information sources

Metadata about information sources (e.g., databases and repositories) can be collected by Query Sampling (QS). Such metadata can include topics and statistics (e.g., term frequencies) about the information sources. This provides important evidence for determining which sources in the distributed information space should be selected for a given user query. The aim of this paper is to find out the semantic relationships between the information sources in order to distribute user queries to a large number of sources. Thereby, we propose an evolutionary approach for automatically conducting QS using multiple crawlers and obtaining the optimized semantic network from the sources. The aim of combining QS and evolutionary methods is to collaboratively extract metadata about target sources and optimally integrate the metadata, respectively. For evaluating the performance of contextualized QS on 122 information sources, we have compared the ranking lists recommended by the proposed method with user feedback (i.e., ideal ranks), and also computed the precision of the discovered subsumptions in terms of the semantic relationships between the target sources. (C) 2010 Elsevier Inc. All rights reserved.