Semi-formal Evaluation of Conversational Characters

被引：15

作者：

Artstein, Ron ^{[1
]}

Gandhe, Sudeep ^{[1
]}

Gerten, Jillian ^{[1
]}

Leuski, Anton ^{[1
]}

Traum, David ^{[1
]}

机构：

[1] Univ So Calif, Inst Creat Technol, Marina Del Rey, CA 90292 USA

来源：

LANGUAGES: FROM FORMAL TO NATURAL | 2009年 / 5533卷

关键词：

D O I：

10.1007/978-3-642-01748-3_2

中图分类号：

TP31 [计算机软件];

学科分类号：

081202 ; 0835 ;

摘要：

Conversational dialogue systems cannot be evaluated in a fully formal manner, because dialogue is heavily dependent on context and current dialogue theory is not precise enough to specify a target output ahead of time. Instead, we evaluate dialogue systems in a semi-formal manner, using human judges to rate the coherence of a conversational character and correlating these judgments with measures extracted from within the system. We present a series of three evaluations of a single conversational character over the course of a year, demonstrating how this kind of evaluation helps bring about an improvement in overall dialogue coherence.

引用

页码：22 / 35

页数：14