A framework for merging and ranking of answers in DeepQA

被引:8
作者
Gondek, D. C. [1 ]
Lally, A. [1 ]
Kalyanpur, A. [1 ]
Murdock, J. W. [1 ]
Duboue, P. A. [2 ]
Zhang, L. [3 ]
Pan, Y. [3 ]
Qiu, Z. M. [3 ]
Welty, C. [1 ]
机构
[1] IBM Corp, Div Res, Thomas J Watson Res Ctr, Yorktown Hts, NY 10598 USA
[2] Labs Foulab, Montreal, PQ H4C 2S3, Canada
[3] IBM Res Div, China Res Lab, Beijing 100193, Peoples R China
关键词
Artificial intelligence;
D O I
10.1147/JRD.2012.2188760
中图分类号
TP3 [计算技术、计算机技术];
学科分类号
080201 [机械制造及其自动化];
摘要
The final stage in the IBM DeepQA pipeline involves ranking all candidate answers according to their evidence scores and judging the likelihood that each candidate answer is correct. In DeepQA, this is done using a machine learning framework that is phase-based, providing capabilities for manipulating the data and applying machine learning in successive applications. We show how this design can be used to implement solutions to particular challenges that arise in applying machine learning for evidence-based hypothesis evaluation. Our approach facilitates an agile development environment for DeepQA; evidence scoring strategies can be easily introduced, revised, and reconfigured without the need for error-prone manual effort to determine how to combine the various evidence scores. We describe the framework, explain the challenges, and evaluate the gain over a baseline machine learning approach.
引用
收藏
页数:12
相关论文
共 19 条
[1]
[Anonymous], IBM J RES DEV
[2]
Atkeson CG, 1997, ARTIF INTELL REV, V11, P11, DOI 10.1023/A:1006559212014
[3]
Robust reductions from ranking to classification [J].
Balcan, Maria-Florina ;
Bansal, Nikhil ;
Beygelzimer, Alina ;
Coppersmith, Don ;
Langford, John ;
Sorkin, Gregory B. .
MACHINE LEARNING, 2008, 72 (1-2) :139-153
[4]
DBpedia - A crystallization point for the Web of Data [J].
Bizer, Christian ;
Lehmann, Jens ;
Kobilarov, Georgi ;
Auer, Soeren ;
Becker, Christian ;
Cyganiak, Richard ;
Hellmann, Sebastian .
JOURNAL OF WEB SEMANTICS, 2009, 7 (03) :154-165
[5]
Chu-Carroll J., 2012, IBM J RES DEV, V56
[6]
Automatic knowledge extraction from documents [J].
Fan, J. ;
Kalyanpur, A. ;
Gondek, D. C. ;
Ferrucci, D. A. .
IBM JOURNAL OF RESEARCH AND DEVELOPMENT, 2012, 56 (3-4)
[7]
Introduction to "This is Watson" [J].
Ferrucci, D. A. .
IBM JOURNAL OF RESEARCH AND DEVELOPMENT, 2012, 56 (3-4)
[8]
Ghosh D., 2010, DSLs in Action, V1st
[9]
Hastie E., 2008, ELEMENTS STAT LEARNI
[10]
Herbrich R, 2000, ADV NEUR IN, P115