Supporting serendipity: Using ambient intelligence to augment user exploration for data mining and Web browsing

Serendipity is the making of fortunate discoveries by accident, and is one of the cornerstones of scientific progress. In today's world of digital data and media, there is now a vast quantity of material that we could potentially encounter, and so there is an increased opportunity of being able to discover interesting things. However, the availability of material does not imply that we will be able to actually find it; the sheer quantity of data mitigates against us being able to discover the interesting nuggets. This paper explores approaches we have taken to support users in their search for interesting and relevant information. The primary concept is the principle that it is more useful to augment user skills in information foraging than it is to try and replace them. We have taken a variety of artificial intelligence, statistical, and visualisation techniques, and combined them with careful design approaches to provide supportive systems that monitor user actions, garner additional information from their surrounding environment and use this enhanced understanding to offer supplemental information that aids the user in their interaction with the system. We present two different systems that have been designed and developed according to these principles. The first system is a data mining system that allows interactive exploration of the data, allowing the user to pose different questions and understand information at different levels of detail. The second supports information foraging of a different sort, aiming to augment users browsing habits in order to help them surf the internet more effectively. Both use ambient intelligence techniques to provide a richer context for the interaction and to help guide it in more effective ways: both have the user as the focal point of the interaction, in control of an iterative exploratory process, working in indirect collaboration with the artificial intelligence components. Each of these systems contains some important concepts of their own: the data mining system has a symbolic genetic algorithm which can be tuned in novel ways to aid knowledge discovery, and which reports results in a user-comprehensible format. The visualisation system supports high-dimensional data, dynamically organised in a three-dimensional space and grouped by similarity. The notions of similarity are further discussed in the internet browsing system, in which an approach to measuring similarity between web pages and a user's interests is presented. We present details of both systems and evaluate their effectiveness. (c) 2006 Elsevier Ltd. All rights reserved.