Adaptive retrieval agents: Internalizing local context and scaling up to the Web

被引:187
作者
Menczer, F [1 ]
Belew, RK
机构
[1] Univ Iowa, Dept Management Sci, Iowa City, IA 52242 USA
[2] Univ Calif San Diego, Dept Comp Sci & Engn, La Jolla, CA 92093 USA
关键词
InfoSpiders; distributed information retrieval; evolutionary algorithms; local selection; internalization; reinforcement learning; neural networks; relevance feedback; linkage topology; scalability; selective query expansion; adaptive on-line Web agents;
D O I
10.1023/A:1007653114902
中图分类号
TP18 [人工智能理论];
学科分类号
081104 [模式识别与智能系统]; 0812 [计算机科学与技术]; 0835 [软件工程]; 1405 [智能科学与技术];
摘要
This paper discusses a novel distributed adaptive algorithm and representation used to construct populations of adaptive Web agents. These InfoSpiders browse networked information environments on-line in search of pages relevant to the user, by traversing hyperlinks in an autonomous and intelligent fashion. Each agent adapts to the spatial and temporal regularities of its local context thanks to a combination of machine learning techniques inspired by ecological models: evolutionary adaptation with local selection, reinforcement learning and selective query expansion by internalization of environmental signals, and optional relevance feedback. We evaluate the feasibility and performance of these methods in three domains: a general class of artificial graph environments, a controlled subset of the Web, and (preliminarly) the full Web. Our results suggest that InfoSpiders could take advantage of the starting points provided by search engines, based on global word statistics, and then use linkage topology to guide their search on-line. We show how this approach can complement the current state of the art, especially with respect to the scalability challenge.
引用
收藏
页码:203 / 242
页数:40
相关论文
共 66 条
[1]
[Anonymous], 1972, Journal of Documentation
[2]
[Anonymous], 1996, Adaptive Individuals in Evolving Populations
[3]
ARMSTRONG R, 1995, AAAI SPRING S INF GA
[4]
AROCENA GO, 1997, P 6 INT WORLD WID WE
[5]
BALABANOVIC M, 1997, P 1 INT C AUT AG
[6]
BELEW RK, IN PRESS FINDING OUT
[7]
Bollacker K. D., 1998, P 2 INT C AUT AG
[8]
SCALABLE INTERNET RESOURCE DISCOVERY - RESEARCH PROBLEMS AND APPROACHES [J].
BOWMAN, CM ;
DANZIG, PB ;
MANBER, U ;
SCHWARTZ, MF .
COMMUNICATIONS OF THE ACM, 1994, 37 (08) :98-&
[9]
Chakrabarti S., 1998, P 7 INT WORLD WID WE
[10]
CHAKRABARTI S, 1998, ACM SIGIR WORKSH HYP