40.1 INTRODUCTION TO QUALITATIVE RESEARCH

40. 1. 1 What Is Qualitative Research?

Qualitative research is a term with varying meanings in educational research. Borg and Gall (1989), for example, suggest that the term is often used interchangeably with terms such as naturalistic, ethnographic, subjective, or postpositivistic. Goetz and LeCompte (1984) choose to use the term ethnographic as an overall rubric for research using qualitative methods and for ethnographies.

In this chapter, qualitative research will be defined as research devoted to developing an understanding of human systems, be they small, such as a technology-using teacher and his or her students and classroom, or large, such as a cultural system. Qualitative research studies typically include ethnographies, case studies, and generally descriptive studies (see 41.2). They often are called ethnographies, but these are somewhat more specific. Goetz and LeCompte, for instance, based on the work of Spradley and McCurdy (1972), define ethnographies as "analytic descriptions or reconstructions of intact cultural scenes and groups" (1984, p. 2). A case study may indeed be viewed as an ethnography; however, the investigator may have set out to answer a particular question rather than to describe a group or scene as a whole.

Qualitative research methods typically include interviews and observations, but may also include case studies, surveys, and historical and document analysis. Case study and survey research are also often considered methods on their own. Survey research (see 37.1, 37.4,41.2.1) and historical and document analysis (see, for instance, 41.2.4) are covered in other chapters in this book; therefore they will not be extensively discussed in this chapter.

Qualitative research has several hallmarks. It is conducted in a natural setting, without intentionally manipulating the environment. It typically involves highly detailed rich descriptions of human behaviors and opinions. The perspective is that humans construct their own reality, and an understanding of what they do may be based on why they believe they do it. There is allowance for the "multiple realities" individuals thus might construct in an environment. The research questions often evolve as the study does, because the researcher wants to know "What is happening," and may not want to bias the study by focusing the investigation too narrowly. The researcher becomes a part of the study by interacting closely with the subjects of the study. The researcher attempts to be open to the subjects' perceptions of "what is"; that is, researchers are bound by the values and world views of the subjects. (For a discussion of self-reflexivity, see 10.4.4.) In qualitative research, it is not necessarily assumed that the findings of one study may be generalized easily to other settings. There is a concern for the uniqueness of a particular setting and participants.

In the following section, we will present some of the many points of debate about the definition and use of qualitative methods.

40.1.2 Comparisons Between Qualitative and Quantitative Methods

Some authors have chosen to posit qualitative and quantitative research as diametrically opposed constructs. This may confuse a beginning researcher in that it simplistically implies that qualitative research might never use numbers while quantitative research might never use subjects' perceptions [Discussion of quantifying qualitative data will follow this chapter, but for an example the reader need only look at the title of Johnson's (1978) introduction to qualitative research design, Quantification in Cultural Anthropology.]

More useful, perhaps, is the comparison of Borg and Gall (1989) who name the two approaches positivistic and naturalistic and compare them on the dimensions of the vision of the nature of reality, relationship of the researcher to the research subject, issues of generalizability, discussion of causality, and role of values.

Lincoln and Guba (1985) and Denzin and Lincoln (1994) define the term paradigm as a systematic set of beliefs, and their accompanying methods, which provide a view of the nature of reality. They contend that the history of inquiry can be divided into eras based on people's view of the world and how to study it. They argue that scientific inquiry is defined by the positivist paradigm, which has prevailed until recently. They call the earliest era the prepositivist era, which included human scientific endeavor about the time of Aristotle to the middle of the 1700s. This was the precursor to a more modern perspective. Lincoln and Guba say that research during this era consisted of passive observation and description. They consider the modern scientific method to have emerged in the positivist era, from about the middle 1700s to the present. Positivism, they note, can be identified by scientific research that involves hypotheses, manipulation, active observation of occurrences, and, thus, testing of hypotheses. These authors argue that the positivist paradigm is limited and is challenged currently by the emerging postpositivist paradigm, which they also call the naturalistic paradigm. [Readers unfamiliar with the evolution of paradigms in research may refer to Kuhn's (1970) seminal work, The Structure of Scientific Revolutions, although Lincoln and Guba (1985) appear to consider Kuhn's views part of the positivist paradigm.]

This conception of the naturalistic paradigm is echoed by Erlandson, Harris, Skipper, and Allen (1993) who note in their book, Doing Naturalistic Inquiry, that naturalistic inquiry is a new paradigm as opposed to the older prevailing positivist one. They say that while naturalistic research may use qualitative research methods, it cannot be equated these methods. They mention the "paradigm wars" raging in research in general. They note that constructivism and naturalistic inquiry have evolved together. [Readers may w refer to Guba's (1990) book, The Paradigm Dialog, in the first few chapters of which these points of view are explored further, as well as Chapters 7 to 10 in this handbook for newer views of educational technology research.]

The paradigm debate as it has evolved in educational technology is more recent. The introduction of critical theory issues (see 9.7), the presentation of qualitative m shops at AECT national conferences, and the discussion of alternative research techniques are all indicators of change (see Yeaman, 1994; Robinson & Driscoll, 1993; Driscoll, 1995; Robinson, 1995). One aspect of the paradigm debate is the issue of how one's perspective directs the type of research questions studied and how methods are chosen. Some believe that researchers must declare a paradigm from which they work, and that the paradigm naturally dictates methods and questions. This is a different approach from that taken in this chapter, namely, that methods may be chosen based on questions to be studied.

Other authors, such as Goetz and LeCompte (1984), contend that it is perhaps not useful to build simplistic dichotomies of research models. They argue that dichotomies such as generative verificative, inductive-deductive, subjective-objective, and constructive-enumerative to describe research models must be examined carefully and that "all factors must be balanced in composing a research design" (p. 48).

While many of the authors above use the term naturalistic inquiry, it is perhaps more useful for that term to be applied to the paradigm as Lincoln and Guba (1985) and Erlandson et al. (1993) apply it. Goetz and LeCompte use the term ethnographic for research using qualitative methods, but ethnography is just one form that qualitative research may take. In this chapter, we will use the term qualitative research. This seems to be a less value-laden term and one that has come to the fore recently. (As evidence, one major publisher of textbooks for social science research, Sage Publications, California, publishes an extensive series of references for all aspects of conducting this type of research under the title "qualitative methods.") It remains to be seen whether this is the term that in decades hence will continue to be used.

In sum, we in this chapter agree that forcing a choice between using qualitative or quantitative methods limits and inhibits the quality of research. Our argument is that the questions a researcher strives to answer should drive the choice of methods. Our assumption is that there is no reason data-gathering methods cannot be combined in a study, that a researcher cannot investigate carefully and creatively any questions he or she chooses (see 11.2.4.2, 39.5.2.2). Rather than limiting our endeavors in this time of tremendous strides in technology development, this approach should enable researchers to take chances, to make leaps, to enhance development in the field by yielding both "answers" and "understanding." As will be seen in the next section, this approach has a solid tradition in educational communications and technology.

That said, given the tremendous ferment in educational research today, it behooves any researcher using qualitative methods to be aware of the "paradigm war" discussions. A researcher may choose to build a study using qualitative methods to answer certain questions, in a study that blends these methods with experimental or quasi-experimental methods. The researcher may design an entirely qualitative study to come to a deep understanding about what is happening in a setting, or how the participants perceive of their world. This study may stand on its own, or be used as a sort of pilot study to generate questions and hypotheses prior to conducting an experimental study. In any case, the researcher should be specific about how he or she defines the assumptions of the study and why what was done was done — in short, to be able to enter into the current and upcoming discussions as a thoughtful, critical, and creative researcher.

40.1.3 How Has Qualitative Research Historically Been Defined in Educational Technology?

In educational communications and technology research, and in educational research in general, there is similar debate about the definition and purpose of qualitative methods. This can be viewed as a natural consequence of discussion in education about the utility of constructivist as opposed to positivist views of education. This discussion can be enjoyed at national and regional conferences in the field, and in the journals. It can be said that the larger debate regarding naturalistic versus positivistic research is creating a more open arena in which studies can be presented and published. Indeed, the editors of the leading journals in the field have indicated they welcome the submission of well-crafted qualitative studies. While not many such reports have been published, it is hoped that this chapter may positively influence the future. We will therefore, in this chapter, avoid the larger debate and focus on qualitative methods as tools.

It may come as a surprise to some that use of qualitative data collection methods has a long tradition in educational technology research. Early research efforts often used qualitative methods to evaluate and describe the use of media in the classroom. Experimental researchers have often used qualitative methods to collect attitude data, for instance, to yield possible explanations of students' behavior. These data are typically collected using surveys, but may be collected using interviews. It is not unusual for an experimental researcher to further inform the study by conducting observations of the subjects. Researchers often conduct a case study to unobtrusively learn more about students, teachers, and trainers who use a new technology. Case studies present detailed data that may be used to derive questions later to be investigated in an experiment. Evaluation researchers have long used qualitative methods, in particular surveys, interviews, observations, and historical and document analysis (see also 34.3, 34.6, Chapter 42).

While not researchers, per se, instructional systems designers have always used the qualitative methods of surveys, interviews, and observations during the front-end analysis and evaluation phases of development. Markle (1989), for example, contends that even in the early, more "behaviorist" days of instructional design, developers listened to their learners, watched them carefully, and humbly incorporated what learners taught them into their drafts of instructional materials. Similarly, what recent authors, especially computer scientists, are calling testing in "software engineering" (Chen & Shen, 1989), "prototype evaluation" (Smith & Wedman, 1988), "prototype testing," "quality assurance" (McLean, 1989), or "quality control (Darabi &_ Dempsey, 1989-90) is clearly formative evaluation, usually incorporating some qualitative methods. Beyond these basic uses of qualitative methods, however, there have been calls in the field to use these methods to address new research questions.

With the increasing use of computer-based interactive technologies in education and industry, opportunities, and at times the responsibility, to explore new questions about the processes of learning and instruction are evolving. Educational technologists have issued the call for the use of more qualitative research methods to explore training and school processes (Bosco, 1986; Clark, 1983; see also 6.5). Driscoll (1995) suggests that educational technologists select research paradigms based on what they perceive as the most critical questions. Noting the debate regarding paradigms, she adds that educational technology is a relatively young field in which "numerous paradigms may vie for acceptability and dominance" (p. 322). Robinson (1995) and Reigeluth (1989) concur, noting the considerable debate within the field regarding suitable research questions and methods. Winn (1989) also calls for more descriptive studies yielding information about learning and instruction. Clark agrees with Winn, calling for reconsideration of how media are studied (1983), and stating that researchers should conduct planned series of studies, selecting methods based on extensive literature reviews (1989). He recommends that prescriptive studies be conducted to determine why instructional development methods work. Qualitative methods can serve these purposes admirably.

The approach taken in this chapter, that choosing qualitative or quantitative methods need not be an either/or proposition, is similar to the approach of Hannafin and his associates (Hannafin & Rieber, 1989; Hooper & Hannafin, 1988) in their development of the ROPES guidelines for designing instruction. Their guidelines blend behaviorist with cognitive principles in what they call applied cognitivism (see 12.4.4).

In our field, new educational technologies are continually being developed. Recent developments have been interactive multimedia, new distance learning systems, information technologies such as hypertext databases and the Internet, interactive learning environments, microworlds, and virtual-reality systems. Many teachers, trainers, administrators, managers, community members, and institutional leaders contend that the evolution of new technologies will continue to change the nature of teaching, training, instruction, and learning (Ambron & Hooper, 1990, 1988; Lambert & Sallis, 1987; Schwartz, 1987; Schwier, 1987; U.S. Congress, OTA, 1988).

It is not only new technologies that require new research methods. The more recent developments in critical theory (see Chapter 9), postmodernism (see Chapter 10), and philosophical thought presented in this handbook and elsewhere (see Yeaman et al., 1994) also suggest distinctive changes and additions to our research endeavors and to the questions and problems in education with which technology is involved.

A recent study that used new technologies and combined qualitative and quantitative methods is that of Wiegmann (1996). In her dissertation, she combined techniques to examine preservice teachers' attitudes and perceptions of efficacy regarding instructional technologies. Her quasi-experimental study used a nonequivalent control-group design to investigate the effects of presentation techniques used in a science methods class. Her data were collected using pre- and postsurveys. She also collected student artifacts in the form of student lesson plans, daily technology-use surveys, and reflective journals focusing on topics pertinent to science education and the use of technology in the classroom. Quantitative data analyses in the form of chi-square and factor analysis were combined with qualitative, more-narrative data about the student journals and lesson plans. Questions included, "Does the method used affect teachers' attitudes toward technology?" and "Does the method used to present preservice teachers with ways to use technology in instruction affect the instructional strategies they incorporate into their lesson plans?"

New technologies also enable researchers to study learners and learning processes in new ways. Computers allow sophisticated tracking of the paths that learners take through a lesson. We can view each decision a learner makes and analyze the relationship among the patterns of those decisions and their performance and attitudes (Dwyer & Leader, 1995).

New technologies may also require that we ask new questions in new ways. We may need to expand our views of what we should investigate and how. For instance, a qualitative view of how teachers and their students use a new technology may yield a view of "what is really happening" when the technology is used. Developers are well aware that instruction is not always delivered as designed, and this holds true for technology-based instruction. The history of educational technology includes records of the failures of a technological approach, often for reasons stemming from poorly planned implementation. We need to know what is really occurring when technologies or new approaches are used. Newman (1989) holds that learning environments can affect instructional technologies. He writes, "How a new piece of educational technology gets used in a particular environment cannot always be anticipated ahead of time. It can be argued that what the environment does with the technology provides critical information to guide design process" (p. 1). He adds, "It is seldom the case that the technology can be inserted into a classroom without changing other aspects of the environment" (p. 3).

A lucid discussion of the issues related to using qualitative techniques in investigating aspects of the technology of computer-based instruction is presented by Neuman (1989). She presents, for example, her findings on teacher perceptions and behaviors for integrating this type of interactive technological innovation into their classrooms. In another qualitative study of an instructional innovation, Jost (1994) investigated aspects of effective use of calculators in teaching calculus (see 12.3, 12.4.4, 14.7, 15.6, 17.7, 19.6, 20.5, 23.5, 23.6, 24.11, 26.7, and 29.7 for discussions of the impact of new technologies and research in educational technology).

40.1.4 Assumptions of This Chapter

Well-designed research is never easy to conduct. Qualitative research studies typically require considerably more time to design, collect, and analyze data and to report the results than do quantitative studies. Yet professors in the field often hear students stating that they plan to do a qualitative study because it will be easier or quicker. Unfortunately, all too often poorly conceived and conducted studies are called qualitative in an effort to avoid defining and describing methods used to collect data, to avoid assumptions of the study, and even to clearly describe results. At conferences, one often hears editors of the leading journals exhorted to publish more qualitative research. Editors reply that they will publish such studies, provided reviewers and editors can determine that the studies are sound and relevant. [See, for example, Smith's (1987) paper signifying that the journal AERJ welcomes the submission of qualitative reports.]

It should be noted that there is still some concern regarding the acceptance of qualitative research by journals. Not every editor or reviewer is an expert in identifying well-developed research reports of qualitative studies. Questions of sample size and validity may be inappropriately raised about qualitative studies, indicating that reviewers may need more experience with qualitative methods, or that reviewers with more experience with qualitative methods could be selected.

The concerns with regard to quality of research are not confined to educational technology. Lincoln and Guba (1985) note that "The naturalistic inquirer soon becomes accustomed to hearing charges that naturalistic studies are undisciplined; that he or she is guilty of 'sloppy' research, engaging in 'merely subjective' observations, responding indiscriminately to the 'loudest bangs or brightest lights' " (p. 289).

Methods for evaluating the soundness of a qualitative study, and for conducting a study ethically, will be presented in a later section. However, before discussing the methods qualitative researchers use, it is critical to illustrate the characteristics of good qualitative research. Not all will be present in any one study, as each study is designed differently to investigate different issues. However, it is worth considering what makes a study "qualitative."

In addition to the characteristics described in the earlier definition of qualitative research, in this chapter many of Lincoln and Guba's (1985) characteristics of naturalistic research will be assumed to apply to qualitative research. Qualitative research is done in a natural setting. The main data-gathering instrument is the human researcher. The researcher uses tacit, that is, intuitive or felt, knowledge, as well as propositional knowledge. Qualitative methods are used generally, but not to the exclusion of quantitative methods. Sampling is often purposive or theoretical rather than random or representative. Data analysis is typically inductive rather than deductive, but again, not exclusively. In naturalistic studies, theory is grounded in the data rather than determined a priori, although in qualitative studies theories often do drive the processes used in the investigation.

In contrast to experimental studies, in qualitative studies the design often emerges as the study progresses, with the researcher continually refining the methods and questions. Similarly, the focus of the study determines what data are collected, and the boundaries of what is studied may change during the study as new issues and questions emerge. In qualitative research, the "reality" or the meaning of a situation and setting are negotiated among the researcher and those studied, with the understanding that multiple realities are always present. Many qualitative studies use a case study approach in the report, rather than a scientific report; some, in fact, describe the results by building a narrative or sort of story. A qualitative researcher tends to interpret results of a study or draw conclusions based on the particulars of that study, rather than in terms of generalizability to other situations and settings. Similarly, such a researcher is likely to be hesitant about advocating broad application of the findings of one study to other settings (Lincoln & Guba, 1985).

A final assumption of this chapter is that qualitative studies can be evaluated for quality, and rigor is not tossed out because a study is not quantitative in nature. While some of the criteria may be different from those used in quantitative research, many criteria for evaluating what Lincoln and Guba call the "trustworthiness" of a qualitative study will be discussed, many related to the particular methods used in qualitative research.

In summary, we concur with the call of Salomon (1991) that it is time to transcend the debate about qualitative versus quantitative research. In a stronger message, Robinson (1995) suggests that, "The paradigm debate should be declared a draw... [We should] accept the dual perspectives of our paradigm debate, if we are to meet the challenges of the future and be at all helpful in shaping the educational success of the next century" (pp. 332-333). Robinson continues, "All ways of knowing and all social constructs should be equally accepted and represented in our literature... individuals should be encouraged to question and consider how they approach the world, how they understand learning, and how they believe knowledge is achieved" (p. 332).