Constructing an index of search goal redefinition through transaction log analysis

Purpose of this paper This paper discusses the methodological aspects of a study, the results of which are detailed in an earlier paper.Methodology Indices of search goal revision (or redefinition) were constructed based on the chronological coding of queries logged on the OCLC First Search service. Queries within search sessions were coded according to their conceptual relationships, based on a set of rules designed to identify possible revisions of search goals, as well as strategic reformulations. Given the same user interface, similar types of user, and large datasets, it as was assumed that the number of strategic reformulations would be reasonably uniform across databases, so that the indices could be regarded as indicators of relative amounts of goal revision.Findings The reliability of the coding used to construct the indices of goal redefinition was tested through parallel coding, and confirmed. Differences between the indices for various databases, on the other hand, were found to be statistically significant, allowing for the conclusion that the indices were able to show differences in the amount of goal redefinition occurring, given the assumption of similar levels of strategic reformulations across databases.Value of paper Transaction log analysis has mostly focused on syntactic issues; few analyses have examined the semantic content of queries or contextualised individual queries as part of an evolving search session. However, this paper shows that although transaction log analysis is a relatively indirect means of examining users' thoughts and intentions during their interaction with an information retrieval system, logs that provide sufficiently rich data may, in certain controlled situations, be used to provide deeper insights into information retrieval behaviour.