42.1 The Nature of Developmental Research

Today, even amid the calls for increased use of alternative research methodologies, the notion of developmental research is often unclear, not only to the broader community of educational researchers but to many instructional technology researchers as well. An understanding of this topic is rooted in the nature of development and research in general, as well as a more specific understanding of the purpose, focus, and techniques of developmental research itself.

42.1.1 The Character of Development

Development, in its most generic sense, implies gradual growth, evolution, and change. This concept has been applied to diverse areas of study and practice. For example, developmental psychology is concerned with human growth. It may concentrate on particular age groups, such as in the areas of adolescent development or life span development. Organizational development is a strategy for changing "the beliefs, attitudes, values, and structure of organizations so that they can better adapt to new ... challenges" (Bennis, 1969, p. 2). Educators are familiar with the notion of professional or staff development. Lieberman and Miller (1992) define this as "the knowledge, skills, abilities, and the necessary conditions for teacher learning on the job" (p. 1045). This same concept is often applied to other professional areas. In the corporate arena, the term executive development also refers to learning processes, and in this setting learning, as a developmental activity, often integrates both classroom instruction and work experience (Smith, 1993).

In the field of instructional technology, development has a particular, somewhat unique, connotation. The most current definition views development as "the process of translating the design specifications into physical form" (Seels & Richey, 1994, p. 35). In other words, it refers to the process of producing instructional materials. Development is viewed as one of the five major domains of theory and practice in the field (see footnote 1). Even though this varies from many other uses of the term development, it is consistent with the fundamental attribute of being a process of growth, and in our field development is a very creative process.

Historically development has been an ambiguous term to many instructional technologists and has generated considerable discussion regarding its proper interpretation. This debate has focused typically on the distinctions between instructional design and instructional development. Heinich, Molenda, and Russell (1993) define instructional development as "the process of analyzing needs, determining what content must be mastered, establishing educational goals, designing materials to reach the objectives, and trying out and revising the program in terms of learner achievement" (p. 445). To many, this is a definition of the instructional systems design (ISD) process.

The confusion is further exacerbated. In 1977, Briggs defined "instructional design" as "the entire process of analysis of learning needs and goals and the development of a delivery system to meet the needs; includes development of instructional materials and activities; and tryout and revision of all instruction and learner assessment activities" (p. xx). In this interpretation, design is the more generic term, encompassing both planning and production. The 1994 definition of the field attempts to clarify these issues by viewing design as the planning phase in which specifications are constructed, and development as the production phase in which the design specifications are actualized (Seels & Richey, 1994). This is not a new distinction (Cronbach & Suppes, 1969; National Center for Educational Research and Development, 1970), even though the past use of the term instructional developer (see Baker, 1973) typically referred to a person who was doing what today we would call both design and development. All would agree, however, that design and development are related processes, and Connop-Scollard (1991) has graphically demonstrated these relationships in a complex chart that identified hundreds of interrelated concepts.

Instructional development traditionally does not include a comprehensive notion of evaluation. For example, the Heinich, Molenda, and Russell definition directly speaks to both formative evaluation of products and programs and needs assessment, but by implication it would also encompass test item construction. Likewise, the majority of instructional design models also address only these aspects of evaluation (Andrews & Goodson, 1980). Notable exceptions are the design models of Dick and Carey (1996) and Seels and Glasgow (1990), both of which include a summative evaluation phase.

However, the word development has a broader definition when it is used within the research context than it has when used within the context of creating instructional products. The focus is no longer only on production, or even on both planning and production. It also includes comprehensive evaluation. As such, developmental research may well address not only formative but summative evaluation as well. It may address not only needs assessment but also broad issues of front-end analysis, such as contextual analysis issues, as conceived by Tessmer and Harris's (1992) environmental analysis, or systemic design issues as described by Richey (1992). When evaluation is approached in a comprehensive manner, the scope of the research effort is often correspondingly expanded to encompass product utilization and management, as well as product creation. Table 42-1 displays the scope of development as discussed in this chapter.

The next step beyond "Utilization & Maintenance" in the schemata of Table 42-1 would be "Impact," the follow-up analysis of the effects of an instructional product or program on the organization or the learner. This type of research typically falls within the scope of traditional evaluation research.

42.1.2 The Character of Research

While research methodologies vary, there are key attributes that transcend the various research orientations and goals. An understanding of these characteristics can shed light on the process of developmental research.

42.1.2.1. The Dimensions of Research. Gay (1987) defines research as "the formal, systematic application of the scientific method to the study of problems" (p. 4). One result of such an effort is the creation of knowledge. Even though research is typically rooted in societal problems, all knowledge produced by research is not necessarily in a form conducive to quick resolution of society's problems. Some knowledge (which is usually generated by basic research) must be specifically transformed to enable its application to a given problem. Other knowledge (which is usually generated by applied research) lends itself to the immediate solution of practical problems. Developmental research clearly falls in the latter category. In this respect, it is similar to other methodologies such as action research.

While the objective of research is to produce knowledge, these products may take a variety of forms. Diesing (1991, p. 325) has noted that:

... the philosophers suggest that social science produces at least three kinds of knowledge: (1) systems of laws, which describe interconnected regularities in society; (2) descriptions, from the inside, of a way of life, community, person, belief system, or scientific community's beliefs; (3) structural models, mathematical or verbal, of dynamic processes exemplified in particular cases.

Research orientations tend to conform to the pursuit of a particular type of knowledge. Experimental research (see 39.1) tends to contribute to the construction of a system of laws. Characteristic of this method is a series of routine checks to ensure self-correction throughout the research, and in the logical positivist tradition, such checks are considered to be rooted in objectivity (Kerlinger, 1964). Qualitative research (see 40.1) primarily contributes to the development of "mirrors for man" so that we can see ourselves better (Kluckhohn, as noted in Diesing, 1991). In the various forms of qualitative research, context and contextual influences become a integral part of the investigation (Driscoll, 1991; Mishler, 1979).

Diesing's third type of knowledge is process knowledge presented in model form. This is usually of great interest to instructional designers and developers, given our history of working with many kinds of process models, such as the graphic models of systematic design procedures and the models of media selection. When inquiry procedures result in this type of knowledge, these endeavors can legitimately be placed in the research realm.

Another traditional characterization of research is as a facilitator of understanding and prediction. In this regard "understanding results from a knowledge of the process or dynamics of a theory. Prediction results from investigation of the outcomes of a theory" (Schwen, 1977, p. 8). These goals can be achieved by either: (1) providing a logical explanation of reality, (2) anticipating the values of one variable based on those of other variables, or (3) determining the states of a model (Dubin, as cited by Schwen, 1977). While these ends, especially the first two, can be achieved through traditional research methodologies, the third is uniquely matched to the goals of developmental research. This was emphasized by Schwen (1977, p. 9):

Inquiry in educational technology may be associated with the planning, implementing, and/or evaluation of the management-of-learning process, where that process employs systematic technological analysis and synthesis ... current definitions of technological process may be found in development models ... (having) the common attributes of (1) disciplined analysis of problem, context, constraints, learners, and task, and (2) disciplined synthesis involving the design of replicable forms of instruction and formative and summative evaluation.

42.1.2.2. The Relationships Between Research and Development. The more typical view of research is the discovery of new knowledge and development as the translation of that knowledge into useful form (Pelz, 1967). This conceptual framework has not only been commonly subscribed to but it was also subsequently extended into the research, development, and diffusion model (Brickell & Clark, and Guba, as cited in Havelock, 1971). Early research methods texts addressed development as the "research and development" process (Borg & Gall, 1971). The processes were separate, though related, dependent, and sequential. In some situations, this orientation still prevails. This view emphasizes development's function of linking practice to research and theory, and recognizes the likelihood that instructional development can highlight researchable problems, and thus serve as a vehicle for stimulating new research (Baker, 1973).
Stowe (1973) has shown the parallels between scientific research and the general methodology of instructional systems design. He notes that both:

• Are objective, empirical problem-solving approaches
• Employ procedural models "couched in the language of mathematics" (p. 167)
  
• Have a predictive power dependent on the degree to which they represent the most critical aspects of reality
  
• Can generate new problems and hypotheses

Nonetheless, Stowe rejected the proposition that systematic design procedures can be viewed as research to a great extent because of the inability of ISD to discover generalizable principles and its intent to produce context-specific solutions. In addition, Stowe cited the distinctions between ISD's orientation toward explanations of "how?" as opposed to research's orientation toward explanations of "why?".

In the contemporary orientation toward research, Stowe's arguments can be interpreted as an overly rigid expression of positivist philosophy. The contextual richness of the typical design and development task increases the likelihood that research on such topics be especially ripe for a qualitative orientation. The ability to provide process explanations exemplifies a type of knowledge production using the Diesing framework, and an avenue to understanding and prediction using Schwen's paradigm. Stowe's arguments drafted over 20 years ago could lead to diametrically opposite conclusions today, provided one accepts the premise that research can have a broader function than the creation of generalizable statements of law. We are taking the position that research can also result in context-specific knowledge and can serve a problem-solving function. This is true of developmental research, as it has commonly thought to be true of evaluation research.

While instructional development typically builds on previous research, developmental research attempts to produce the models and principles that guide the design, development, and evaluation processes. As such, doing development and studying development are two different enterprises.

42.1.3 The Background of Developmental Research (see footnote 2)

The field of instructional technology as it exists today emerged primarily from a convergence of the fields of audiovisual education and instructional psychology (see 1.5, 2.3, 5.2). In audiovisual education the emphasis was on the role of media as an enhancement of the teaching/learning process and an aid in the communication process, and there was much interest in materials production. On the other hand, in instructional psychology the nature of the learner and the learning process took precedence over the nature of the delivery methodology, and there was much interest in instructional design. Complementing the instructional psychology roots was the application of systems theory to instruction (see 3.2) that resulted in the instructional systems design movement (Seels & Richey, 1994). This conceptual and professional merger came to fruition in the 1960s and 1970s. During this period, instructional design and development came to assume the role of the "linking science" that John Dewey had called for at the turn of the century (Reigeluth, 1983).

Not surprisingly, it was during this same period that the term developmental research emerged. This new orientation was exemplified by the shift in topics between the First and Second Handbooks of Research on Teaching (Gage, 1963; Travers, 1973). In the 1963 handbook, media was addressed as an area of research with a major emphasis on media comparison research, and all research methodologies considered were quantitative. In the 1973 handbook, media continued to be included as a research area, but the research methodologies were varied, including Eva Baker's chapter on "The Technology of Instructional Development." This chapter describes in detail the process of systematic product design, development, and evaluation. Of significance is the fact that the entire methodology section was titled "Methods and Techniques of Research and Development."

This was a period in which federal support of educational research mushroomed. Regional research and development laboratories were established, and the ERIC system was devised for dissemination. Clifford (1971) estimated that appropriations for educational "research and development for 1966 through 1968 alone equaled three-fourths of all funds ever made available" (p. 1). Research-based product and program development had become firmly established as part of the scientific movement in education. At this time, Wittrock (1967) hailed the use of empirical measurement and experimentation to explain product effectiveness. Such activities "could change the development of products into research, with empirical results and theory generalizable to new problems" (p. 148).

Hilgard (1964) characterized research as a continuum from basic research on topics not directly relevant to learning through the advocacy and adoption stages of technological development. Saettler (1990) maintained that the last three of Hilgard's research categories were directly within the domain of instructional technology. These included laboratory, classroom, and special teacher research; tryout in "normal" classrooms; and advocacy and adoption. Note that these are portrayed as types of research, rather than applications of research, and they are all encompassed within the framework of developmental research.

While instructional technology is not the only field concerned with learning in applied settings, few would dispute the critical role played by these three types of research in our field. Moreover, our uniqueness among educational fields is not only our concern with technology, but rather our emphasis on the design, development, and use of processes and resources for learning (Seels & Richey, 1994). Given this definition of the field, developmental research is critically important to the evolution of our theory base.

42.1.4 The Character of Developmental Research

The distinctions between "doing" and "studying" design and development provide further clarification of developmental research activities. These distinctions can be described in terms of examining the focus, techniques, and tools of developmental research.

42.1.4.1. The Focus of Developmental Research. The general purposes of research have been described as knowledge production, understanding, and prediction. Within this framework, developmental research has particular emphases that vary in terms of the extent to which the conclusions are generalizable or contextually specific. Table 42-2 portrays the relationships between the two general types of developmental research.

The most straightforward developmental research projects fall into the first category of Table 42-2. This category typically involves situations in which the product development process used in a particular situation is described and analyzed, and the final product is evaluated, such as Buch's (1988/1989) documentation of the development of an industrial microcomputer training program. Driscoll (1991) called this research paradigm systems-based evaluation. Some Type 1 developmental studies reflect traditional evaluation orientations in which the development process is not addressed, and only the product or program evaluation is described. An example of this type of study is O'Quin, Kinsey, and Beery's (1987) report of the evaluation of a microcomputer training workshop for college personnel. Regardless of the nature of the Type 1 study, the results are typically context and product specific, even though the implications for similar situations may be discussed.

The second type of developmental study is oriented toward a general analysis of either design, development, or evaluation processes as a whole or any particular component. They are similar to those studies Driscoll (1991) calls model development and technique development research. While there are fewer studies that focus on the more global orientation, Taylor and Ellis's (1991) study did so by evaluating the use of instructional systems design in the Navy, and Kress (1989/1990) did so by comparing the impact of systematically designed training with a nonsystematic approach. Other studies in this category focus on only one phase of the design/development/evaluation process, such as Jonassen's (1988) case study of using needs assessment data in the development of a university program. Type 2 research may draw its population from either one target project such as King and Dille's (1993) study of the application of quality concepts in the systematic design of instruction at the Motorola Training and Education Center, or from a variety of design and development environments. Examples of the latter approach include Riplinger's (1985/1987) survey of current task analysis procedures, and Cambre's (1978/1979) historical study of formative evaluation in instructional film and television. Typically, conclusions from Type 2 developmental research are generalized, even though there are instances of context specific conclusions in the literature.

42.1.4.2. Nondevelopmental Research in the Field. A critical aspect of any concept definition is the identification of nonexamples as well as examples. This is especially important with respect to developmental research, since it often seems to overlap with other key methodologies used in the field. Even so, developmental research does not encompass studies such as the following:

• Instructional psychology studies
• Media or delivery system comparison or impact studies
  
• Message design and communication studies
• Policy analysis or formation studies
  
• Research on the profession

While results from research in these areas impact the development process, the study of variables embedded in such topics does not constitute developmental research. For example, design and development is dependent on what we know about the learning process. We have learned from the research literature that transfer of training is impacted by motivation, organizational climate, and previous educational experiences. Therefore, one may expand a front-end analysis to address such issues, or even construct design models that reflect this information, but the foundational research would not be considered developmental. If the new models were tested, or programs evaluated that were designed using such models, this research would qualify as developmental.

A fundamental distinction should be made between reports that analyze actual development projects, and descriptions of recommended design and development procedural models. While these models may represent a synthesis of the research, they do not constitute research in themselves. A good example of this latter situation is Park and Hannafin's (1993) guidelines for designing interactive multimedia. These guidelines are generalized principles that speak to the development process, and they are based on a large body of research. Nonetheless, the identification and explanation of the guidelines is not in itself an example of developmental research. The instructional technology literature includes many examples of such work. They often provide the stimulus for a line of new research, even though these articles are not considered to be research reports themselves. There are many examples today of such work, including explorations of topics such as cognitive task analysis (Ryder & Redding, 1993), or the nature of design and designer decision making (Rowland,1993).

42.1.4.3. The Techniques and Tools of Developmental Research. Developmental researchers employ a variety of research methodologies, applying any tool that meets their requirements. Summative evaluation studies often employ classical experimental designs. Needs assessments may incorporate qualitative approaches. Process studies may adopt descriptive survey methods. Even historical research methods may be used in developmental projects.

Traditional research tools and traditional design tools facilitate the developmental endeavor. Expertise is often required in statistical analysis, measurement theory, and methods of establishing internal and external validity. Likewise, the developmental researcher (even one studying previously designed instruction) requires a command of design techniques and theory. Additional design proficiency is frequently required when using electronic design systems and aids, conducting environmental analyses, and defining ways to decrease design cycle time.

A developmental research project may include several distinct stages, each of which involves reporting and analyzing a data set. Merely conducting a comprehensive design and development project does not constitute conducting a developmental research project, even using its most narrow Type 1 definition. One must also include the analysis and reporting stage to warrant being classified as developmental research.

Developmental research projects may include a number of component parts. Substudies may be conducted to analyze and define the instructional problem, to specify the content, or to determine instrument reliability and validity. Substudies may be conducted to provide a formative evaluation, a summative evaluation, or a follow-up of postinstruction performance. Consequently, reports of developmental research are frequently quite long, often prohibiting publication of the full study.

Reports of developmental projects can often be found in:

• Professional journals
  
• Doctoral dissertations
  
• Educational Resource Information Center (ERIC) collections of unpublished project reports
  
• Conference proceedings

The nature of the reports varies depending on the dissemination vehicle. Sometimes, full developmental projects are split into more easily publishable units (or even summarized) to facilitate publication in the traditional research journals. Developmental research reports are also published in practitioner-oriented journals and magazines, and the methodology and theoretical base of the studies is omitted to conform to the traditions of those periodicals. The next section will further define the nature of developmental research by summarizing studies that are representative of the wide range of research in this category.

Footnotes:

1. In addition to development, the domains also include design, management, utilization, and evaluation. Brief histories of each of these domains can be found in Chapter 2 of Seels and Richey (1994)

2. A history of instructional development can be found in Baker (1973), which primarily summarizes the work in research-based product development from the turn of the century to 1970. Baker, however, does not address developmental research as it is presented in this chapter.