25.6 INTEGRATING USERS, ACCESS, AND INFORMATION: THREE LONGITUDINAL STUDIES

Research suggesting linkages between users, access, and information supplies the meaning to information access. Examining each of the research areas of users, access, and information is one way to compare and contrast the existing state of knowledge about information access. However, at some point, all three areas need to be integrated into a meaningful process. Finding out that something exists, one definition of access, does not imply that the information is meaningful or useful. Access does not even necessarily imply retrieval of the information. In a limited sense, access might simply indicate that a given item or piece of information exists. In this section, three long-term research projects that examine the relationships between users, access, and information are examined in detail, namely the work of:

1. Tefko Saracevic and associates on information seeking and retrieval
2. Brenda Dervin and associates on sense making in information access
3. 3. Carol Kuhlthau on the information search process

These projects are similar in that they are longitudinal; they address all three aspects of information access discussed in this chapter; and they focus on the process of information access. They are substantially different in methodology, theoretical orientation, process focus, and population studied. Taken together, these three research projects represent the most substantive and influential work on information access in LIS.

25.6.1 Information Seeking and Retrieval

The information-seeking and retrieval project of Tefko Saracevic and associates discussed below began in 1980 with the development of a model and a research methodology (Derr, 1982, 1984, 1985a, 1985b; Pao, 1983; Saracevic, 1980, 1983, 1984, 1985). The model has five components: user, question, searcher, search, and items retrieved. Each of these components is measured in multiple ways, including Likert scaling, categorical variables, standardized tests, and numbers of items. The methodology is quantitative and uses traditional regression and analysis of variance techniques.

However regression analysis and analysis of variance proved to be disappointing in uncovering significant relationships" (Saracevic, Kantor, Chamis & Trivison, 1988, p. 213). The use of the cross-product ratio, traditionally found in biomedical research, was a more powerful statistical method for application in LIS research on information seeking and retrieval. The results of the application of this technique are detailed in the study described below.

Between 1985 and 1987, Saracevic et al. (1987, 1988) conducted an extensive research study on the model's five components and obtained significant results using the cross-product ratio statistic. The study consisted of 40 research questions submitted by 40 different academic and industrial workers. These questions were researched by 36 experienced users of DIALOG. In addition, the same 40 questions were researched in four different ways by three project researchers. This resulted in a total of 200 outside searches on the 40 questions, and 360 project searches on the 40 questions. A total of 5,411 unique items were identified as solutions to the problems posed in the 40 questions. Of these, 1,343 were judged as relevant, 1,448 as partially relevant, and 1,620 as not relevant by the original question askers.

Each of the five components of the model was operationalized with multiple measures. Use of the cross-product ratio resulted in significant effects for the following aspects of the model's components:

• Of the four measures for users (problem, intent, public knowledge, and internal knowledge), only problem and public knowledge showed a significant positive impact on retrieving relevant items.
• Of the five measures for questions (domain, clarity, specificity, complexity, and presuppositions), low clarity, low specificity, high complexity, and many suppositions showed a significant positive impact on retrieving relevant items.
• Of the four measures for searchers (Remote Associations Test-RAT, Symbolic Reasoning Test-SRT, Learning Style Inventory-LSI, and search experience), only high RAT scores and high Abstract Conceptualization on the LSI showed a significant positive impact on retrieving relevant items. Concrete Experience on the LIS was negatively related to retrieving relevant items.
• Of the six measures of search tactics and efficiency (commands, iterative cycles, search terms, preparation time, on-line time, and total time), high numbers of cycles, low numbers of search terms, below-average preparation time, and below-average total time showed a significant positive impact on the retrieval of relevant items.
• Of the six indicators of item retrieval (relevant, partially relevant, not relevant, evaluated items, not evaluated items, and total items), if the number of relevant, partially relevant, and total items were high, then there was a significant positive impact on the retrieval of relevant items.

In addition to examining relationships to relevance, this study also looked at all categories described above as related to precision and recall. These measures are both traditionally used in overall macrowise retrieval research. While there were some significant findings for these categories, the more important information was gained from the microwise item analysis discussed above related to relevance. This study made no attempt to aggregate the effects of the variables, which showed positive relationships to relevance, precision, and recall.

One of the most significant results of this study was from pairwise comparisons of all 200 outside searches done for the 40 questions. This resulted in 800 pairs of comparisons examining the overlap of terms selected for searching and items retrieved by searchers. The overlap of search terms ,was relatively low. In 56.4% of the cases, the agreement was less than 25% (Saracevic & Kantor, 1988a, p. 204). In item retrieval, 69% of the items were retrieved only once. If an item was retrieved more than once, it was more likely to be relevant. The odds of an item retrieved once being relevant was 10:10. The odds of an item retrieved by five different searchers being relevant was 57: 10.

This research project has implications for all three areas of information access discussed in this chapter: users, access, and information. In terms of access, the project demonstrates the importance of questions and users on the development of access points and interfaces. Interactions-whether between a search intermediary as described in this study or a system such as an OPAC, an expert system, or an on-line database require sophistication in interpreting users' questions. The use of keywords from a question was the least effective search method found in this study. The most effective method was information compiled from a number of searchers using different terms and search strategies.

In terms of information, this study clearly demonstrated the lack of overlap in the use of search terms and the retrieval of relevant information. This has strong implications for indexing, database construction, and information organization. There is not a simple correspondence in choosing an exactly correct term or series of terms to solve a complex problem. Information structures that restrict the ability of searchers to create complex and iterated searches reduce the possibilities for effective information problem solving. .

In terms of users, implications from the searcher's strategies can be applied to developing instruction for individual users as well as professional searchers. The use of many search terms and concrete linear strategies resulted in retrieving fewer relevant items. Focusing instruction on developing word association skills and abstract conceptualizations of information problems could improve user's solutions to information problems and increase feelings of satisfaction.

The Saracevic information-seeking and retrieval project developed a model and implemented a quantitative statistic new to information science research: the cross-product ratio. Testing the model is a step toward theory building. The project created hypotheses and questions that need to be tested further and used to develop an information-seeking and retrieval theory. Saracevic and Kantor state in the conclusion of the 1988 report that the theory still needs to be developed. They suggest the need for a theory of information seeking and retrieving that has the following characteristics:

• Be part of a broad theory of human information behavior
• Not use computer logic as a model because it is inadequate
• Incorporate both context and content of information in addition to syntax and logic
• Incorporate individual differences in patterns
• Be dynamic and interactive rather than linear and unidimensional
• Account for both human-human and human-computer interactions
• Allow for dynamic changes over time (Saracevic & Kantor, 1988b, p. 213)

The longitudinal research projects of Brenda Dervin and associates (Atwood & Dervin, 1982; Dervin, 1983, 1989a, 1989b; Dervin & Clark, 1987; Dervin, Johnson & Nilan, 1982) uses both qualitative and quantitative measures. It is based in communication theory and examined populations of different ethnicities, age groups, and sociocultural backgrounds. The studies have been performed with other information agencies as well as libraries. Dervin's work focuses on process and identifies the user as the foundational element in that process.

In the literature, this body of work is generally called sense-making theory. Dervin describes her work in this way: "Some people call sense making a theory, others a set of methods, others a methodology, others a body of findings. In the most general sense it is all of these" (Dervin, 1992, p. 61).

Dervin's sense-making research covers 40 projects over 17 years (Dervin, 1992). They include Asian, Hispanic, Afro-American, and Anglo populations, as well as teenagers, preschoolers, and adults. Subjects range from blood donation to politics to completing class assignments. Information agencies include libraries, health organizations, and government. bodies. These series of -projects are quite possibly the broadest and most wide ranging in LIS related to information access.

The foundation of sense making is a series of assumptions about information and communication. Sense making is founded on the assumption of discontinuity: discontinuity in time, in space, between people, and within an individual. Dervin conceptualizes this discontinuity as a gap. In information access, the goal is for the individual to create a bridge from the problem situation and over the gap through the use of information resources. This process is depicted as a triangle, with the points being situation, gap, and helps (or resources). Sense making can be applied to intrapersonal, interpersonal, small group, organizational, telecommunications, databases, societal, or other information use and information-seeking environments.

Three aspects of information drive both sense-making theory and the implementation of specific methods:

1. Information is constructed by the individual.
2. Information use, therefore, must center on the perspective of the actor.
3. Processes and behavioral strategies and tactics are the focus of research.

Methods develop from these three foundations in the following ways. A person constructs information through a series of steps to bridge gaps. A discontinuity occurs when the step stops. This is the point at which the researcher needs to acquire information about what processes and strategies are being used by the individual. Dervin has developed a set of interview techniques to help understand and obtain appropriate information. The foundational interview method is called the micro-moment time-line interview. A respondent is asked to recall what happened (the situation), what the gap was, and what the goal was (or the help that was needed). This type of interview results in detailed, time-focused information related to all points of the sense-making triangle. Three other interview techniques have also been developed but are not as intensive and focus on more specific issues. The abbreviated time-line interview is used to concentrate on one point of the triangle and elicit specific information about that factor. The help chain looks at the help point of the triangle. The question is asked: How did the library, the book, the video, etc., help? With each response, the interviewee is asked again: And how did that help? The question is repeated until the respondent feels his or her answer is completed. Message queuing is the third focused interview technique. Respondents focus specifically on the gap-bridging or gap-defining portion of the sense-making environment.

Information gathered from the interviews is analyzed in both qualitative and quantitative ways. One of the primary methods, however, is content analysis. Results of studies provide data for both hypothesis testing of aspects of sense-making theory and data to inform and help solve the problem of practice on which the research study is based. Dervin's goals are both the development of theory and the solving of problems of practice. Each is equally important to the use of sense making (Dervin, 1992).

In a study for the California State Library (Atwood & Dervin, 1982), content analysis was used to classify how individuals identified and described gaps. The hypothesis tested was that the nature of a gap would be consistent across race. It was found that race did not predict information seeking in terms of questions asked. Race did interact with the channels and sources individuals used to get answers. The practice component of the California study provided descriptive information on need, questions, barriers, strategies, and successes of the people interviewed to help the California State Library develop and deliver services of value to users and potential users of all races.

A theory-building result across studies was the identification and naming of categories of stops, gap definings, and helps. Stops included such components as barriers, spinouts, and washouts. Gap definings have related to self, others, objects, actions, and events. Help categories included creating ideas, acquiring skills, calming down, and reaching goals. These categories were consistent across populations and problem situations.

Dervin's research has implications for all three areas of information access discussed in this chapter. In terms of users, this is a strong, theoretically based explanation of how users access information, deal with problems of access, and identify helps and resources. It provides the necessary foundation to develop hypotheses within a framework and test those hypotheses about users with validated and reliable methods that are directly cogent to the theory.

In terms of access, the results of these studies have implications for the design and development of user interfaces and human reference interactions. One of the particularly interesting cross-situational and cross-participant stops identified is the idea of someone (or something) leading the individual down a road they do not wish to travel. No matter how correct that road may seem to the computer, the reference librarian or the teacher who is trying to lead it becomes a stop to the individual with the problem. The need for flexibility and using the paths of the questioner should be integrated into human-human interactions and human-computer interactions.

In terms of information, an example from a study of blood donors (Dervin et al., 1982) demonstrates information issues that were addressed by the sense-making research. A series of different predictors were compared to see how donors wanted information to help them. They wanted and needed different information at different steps in the donation process. Sometimes they wanted details. Sometimes they wanted reassurance and no details. Different donors might also want information from different sources. To the question, "Will I faint?" one donor might want a doctor's response, while another might want statistical counts of how many people faint, and a third might want information from a previous donor. The solution in this instance was to construct a computerized question-answering system that could be accessed at the five points identified by donors as gaps or stop points. Typical questions were displayed on the screens, and a variety of question-answering strategies was made available, including responses from various sources such as doctors, statistics, and previous donors.

The research projects of Carol Kuhlthau (Kuhlthau, 1983, 1988, 1989, 1993a, 1993b, 1993c) were qualitative and resulted in the creation of grounded theory about the information search process. The research looks at information access in instructional environments. The purposes of Kuhlthau's work are to develop a theory of the information search process and to provide guidelines for instruction in this process (Kuhtthau, 1993c). This research concentrates on process and identifies the user as the foundational element in that process.

Kuhlthau's (1983) research began as a test of Kelly's personal construct theory. Kuhlthau explored the stages of the information search process and believed that "... the search process is, in itself, a process of construction" (Kuhlthau, 1988, p. 233). All of Kuhlthau's studies use field-based qualitative research methods to build grounded theory about the information search process. The original high school study (1983) was followed with further studies varying the population and refining the theory. Across-studies (Kuhlthau, 1989) looked at high school students, college students, high-, middle-, and low-achieving high school seniors, and academic and public library users. The final theory is a three-by-seven matrix of stages by process. The three states are feelings, thoughts, and actions. The seven stages are: task initiation, topic selection, prefocus exploration, focus formulation, information collection, search closure, and starting writing. For each of the three stages, a continuum of activities is developed along the seven processes. For example, feelings range from uncertainty to relief, thoughts from ambiguity to specificity, and actions from seeking relevant information to seeking pertinent information.

Kuhlthau used these studies and the grounded theory to create a principle of uncertainty for information seeking (1993a). This principle has six corollaries and seeks to integrate the cognitive. processes of information seeking with affective considerations. In the 1993a article, Kuhlthau clearly indicates the constructivist focus of her information-seeking research and examines the influences of Dewey, Kelly, and Bruner on the development of the principle's conceptual framework. The basic uncertainty principle represented by this research is stated as: "Uncertainty is a cognitive state which commonly causes affective symptoms of anxiety and lack of confidence" (Kuhlthau, 1993a, p. 347). The six corollaries to this principle are: the process corollary, the formulation corollary, the redundancy corollary, the mood corollary, the prediction corollary, and the interest corollary. This principle and the associated corollaries are a strong conceptual framework for further research in information needs and uses.

Kuhlthau's research has implications for all three areas of information access discussed in this chapter- users, access, and information. In terms of users, Kuhlthau's work is highly focused on the impact of mood and internal affect on the process of seeking and finding information. She clearly delineates cycles that users can refer to and gauge their own feelings. The user's feelings of helplessness or frustration can be acknowledged as part of the infomationseeking process.

Both Dervin and Saracevic also addressed affective issues. Saracevic looked at user satisfaction with items received to answer questions posed. Dervin examined users' perceptions of institutional images. Dervin also considered feelings, emotions, and affect as part of the process that may be gap defining or barrier producing. The differences between Saracevic, Dervin, and Kuhlthau are in degree of focus. Kuhlthau's work relies the most heavily on the impact of feelings on behavior.

In terms of access, Kuhlthau's grounded theory provides observations on the information-seeking process. It appears that information may not be as important as an individual's state of mind at the time the information is accessed. Information accessed early in the seeking process may be considered too difficult, not relevant, or unintelligible. At later stages of the seeking process, the same information, from the same access point, under the same delivery conditions, could be perceived differently. In designing access systems, the different states of users when they access the system, either human or machine, need to be taken into consideration.

In terms of information, the-notion of value-free information comes into play. Often databases, indexing systems, and other forms of information organization assume that information can be sorted into value-free categories and structures. Individuals carry values and feelings into the information search process. Those beliefs interact with information structures no matter how they were constructed. A related idea from Kuhlthau's work is the difference between relevant information and pertinent information. Most information structures are designed to provide relevant but not necessarily pertinent information. Pertinence is a measure of value.

25.6.4 Comparing Three Longitudinal Information Access Projects

Each of the three projects discussed above has applications to user, access, and information research. They each use distinct methodologies, focus on process, and take the user as their center of study. In addition, these researchers have continued their projects over 10 years or longer. There is longevity and growing complexity in their ideas and theoretical positions. They have set ideas out, tested them, and refined and revised.

These three projects demonstrate three different organizational structures for conducting research. Dervin's projects include both qualitative and quantitative methods and are based in communication theory. Kuhlthau's qualitative methods and grounded theory provide an examination of personal and affective characteristics. Saracevic demonstrates how a quantitative method from another research area can be successfully adapted for use in LIS. All three projects use constructivist assumptions in thinking about users' behaviors and beliefs. They acknowledge the role of the user in constructing information. They also demonstrate that quantitative, qualitative, and a combination of quantitative and qualitative methods can all be successfully used to understand differing aspects of information access.

Taken together these three longitudinal research projects suggest the range of theory use, methodological differences, and organizational foundations that can be applied to studying the overall process of information access, including users, access, and information.