On Federated Search UI

It's a very ancient saying,
But a true and honest thought,
That if you become a teacher,
By your pupils you'll be taught.(1)

The daunting task of finding a needle in a haystack has never been easier. Or, so it appears. The web and in turn Google, Yahoo and other search engines (to which I will refer as Yogles) liberated a previously untapped human desire to locate obscure bits of information for personal consumption. Suddenly, millions of people are researching not only best airfare, pet rocks, organically correct coffee beans and lovers, but also complex medical conditions, dietary supplements or the secret life of Barbie and Ken. And the amazing thing is… this stuff can be found!!!

An activity that used to be limited to academic and research settings and required years of training and specialized knowledge is now reduced to the phrase “Google it!” This is probably good for Human Kind, but if you are developing sophisticated federated (or meta) search software you are in trouble. While scientific search still requires expertise to navigate the exponentially growing sea of knowledge, users increasingly demand the task to be as simple as Yogle makes it appear.

The user interface problem for the developers of federated search tools can be described using the agricultural haystack metaphor. Finding a needle in a haystack can be easy if you are equipped with the right tool, say a powerful magnet. This is what happens with Yogles--the magnets. MetaSearch tools try help with a more complicated task. It involves searching for a particular piece of hay in the haystack; clearly, a magnet is not as effective. Here, we face a challenge of a higher magnitude.

Developers spend a great deal of effort and imagination on creating fantastic search engines. But these, like all engines are buried under the hood, and most of the work done by the tool is transparent to the user with the exception of speed. The users, on the other hand, care little about HOW the search is done, and more about WHAT is found. The Yogle user interface underscores this gap: a single search field is very easy to use but the results often include thousands of records, and there are absolutely no organizing tools to manage a retrieved set or past results.

The rational for this minimalist approach to the user interface may be that a particular engine can be so good that what you are looking for would be displayed on the first screen and even in the first line. The problem is that the information displayed on the first page or line is often based on what OTHERS were looking for. In other words, it is easier to search for a coin under the light post, but what if the coin you are after is on page 75,233 out of 270,008?

By becoming a de facto standard for searching, Yogle’s user interface inflicts users with high expectations for ease of use and satisfaction in accomplishing a search task, even if the search task they need to perform is as remote in complexity and scope from the common search as an Amoeba is remote from Humans. Moreover, while the starting point of the search process has been popularized and has now become familiar to millions, we still lack widely accepted interaction patterns for organization and management of large sets of retrieved data.

In order to build a successful user interface for a federated search tool, developers would benefit from understanding the mental models that users create in order to deal with the entire search process. The process of profiling users--building use-case libraries that describe user types, scenarios of use, goals, workflows and outcomes--is common to software development. The problem with modeling for federated search UI is that many of the users themselves lack a clear mental model of what search, retrieval and analysis mean! In fact, a wide majority of users may not know what a federated search means.

Most software applications enable users to perform familiar tasks. As such, software represents the next level in the evolutionary ladder of productivity. For example, the word processor replaced the typewriter, and both advanced letter writing from the use of the pen, pencil, and feather. The introduction of e-Mail further reduced delivery speed and made the use of paper optional. Despite all these changes, the task itself--communicating in writing has changed little since its inception. Users are able to perform with software familiar tasks that have predictable outcomes. When the user’s perception of the task is closely replicated in the user interface, the application is perceived to be intuitive.

For most users, however, meta searching electronic data is not a familiar task. Users have to negotiate a lot of new technical information, such as terminology, scope of search (where are we searching), the impact of logic (AND, OR, NOT) on the result set, how ranking is determined and so on. Further, the absence of established workflows and interaction metaphors complicates mapping federated search activities.

The Yogle search engines excel in mapping to a common search structure. They work well when we know what we are looking for, but just don’t know where to find it. The challenge – and yes – the opportunity for developers of meta search tools is to come up with a successful user interface metaphor that encompasses the complete discovery process and reflects a wide range of user skills and expertise in searching for knowledge. The user interface of federated search tools requires a revolutionary rather than an evolutionary change, and it can only be achieved through collaboration among developers, librarians, patrons, and other stakeholders. We may even consider changing the name for this class of software to “MetaDiscovery Tools” to project a notion of a complete process that covers the search-retrieve-analyze-manage cycle.

Footnotes
1) From the soundtrack lyrics for ‘Getting to know you’ in the movie ‘The King and I’

Note to the reader: This is a reprint of my talk at ALA summer 2004, Orlando, LITA Pre conference Friday, June 25, 2004 Usability Issues in MetaSearch Interface Design