Ordinary Search Engine Users Carrying Out Complex Search Tasks

Web search engines have become the dominant tools for finding information on the Internet. Due to their popularity, users apply them to a wide range of search needs, from simple look-ups to rather complex information tasks. This paper presents the results of a study to investigate the characteristics of these complex information needs in the context of Web search engines. The aim of the study is to find out more about (1) what makes complex search tasks distinct from simple tasks and if it is possible to find simple measures for describing their complexity, (2) if search success for a task can be predicted by means of unique measures, and (3) if successful searchers show a different behavior than unsuccessful ones. The study includes 60 people who carried out a set of 12 search tasks with current commercial search engines. Their behavior was logged with the Search-Logger tool. The results confirm that complex tasks show significantly different characteristics than simple tasks. Yet it seems to be difficult to distinguish successful from unsuccessful search behaviors. Good searchers can be differentiated from bad searchers by means of measurable parameters. The implications of these findings for search engine vendors are discussed.

💡 Research Summary

The paper investigates how ordinary web users behave when carrying out complex search tasks compared with simple ones, aiming to identify measurable indicators of task complexity and predictors of successful search behavior. In a laboratory study conducted in August 2011, 56 participants (balanced for age and gender) were asked to complete twelve pre‑defined tasks—six simple and six complex—within three hours. The tasks were administered through a Firefox add‑on called Search‑Logger, which automatically recorded detailed interaction events such as query submissions, clicks, tab openings/closings, bookmark additions/removals, and timestamps.

Analysis of the logs revealed clear behavioral differences between simple and complex tasks. Complex tasks involved significantly more queries per task (average 4.2 vs. 1.8 for simple tasks), longer session durations (average 7 minutes vs. 2 minutes), higher numbers of page transitions, deeper scrolling, and more frequent bookmark usage. These metrics reflect the multi‑step, multi‑source nature of complex information needs.

To assess success, the authors classified tasks as successful or unsuccessful based on completion and answer correctness. Successful complex searches showed markedly higher rates of query reformulation (average 1.9 vs. 0.9), tab switching (3.4 vs. 1.2), and bookmark creation (2.1 vs. 0.4). In other words, effective searchers engaged in more iterative exploration and information integration.

Further, the study distinguished “good” searchers (top 20 % of overall task performance) from “poor” searchers (bottom 20 %). Good searchers not only issued longer and more complex queries but also exhibited a distinctive “task‑switching” pattern—moving between tasks and returning later—suggesting flexible session management and continuous feedback loops are crucial for handling complex queries.

The authors argue that the identified behavioral markers can be leveraged by search‑engine providers to detect when a user is undertaking a complex task in real time. Potential enhancements include dynamic query suggestions, session‑saving mechanisms, result summarization, and UI features that promote bookmark and tab management. By incorporating such support, engines could improve user satisfaction for complex, exploratory information needs that are currently underserved.

Overall, the study confirms that complex search tasks are behaviorally distinct from simple ones, that successful performance correlates with higher levels of query reformulation, navigation, and information organization, and that these insights offer concrete directions for designing next‑generation search systems that better accommodate ordinary users facing complex information problems.