19.4 THE 20-YEAR HISTORY OF ITS

Instead of discussing individual tutoring systems that spanned this period, we present salient characteristics of systems appearing at various points in time, illustrating with exemplar tutors. For excellent discussions of individual intelligent tutoring systems, see the following books: Bierman, Breuker, Sandberg, 1989; Goodyear, 1991; Lajoie & Derry, 1993; Lawler & Yazdani, 1987; Nickerson and Zodhiates, 1988; Polson & Richardson, 1988; Psotka, Massey, Mutter, 1988; Regian & Shute, 1992; Self, 1988; Sleeman & Brown, 1982, and Wenger, 1987. The issues, by decade, that will be discussed can be seen in table 19-1.

19.4.1 Up through the 1970s: Defining the Issues

Hardware and software have evolved at an astounding rate over the past 20 years. To put things in perspective, consider the 1970s--"Pong" was the rage (i.e., a simple black-and-white computerized table tennis game) and 8K random access memory (RAM) the norm for a PC. Computer-administered instruction developed before the 1970s was inflexible and didactic because the systems had very limited capabilities (i.e., memory capacity and computational speed) for adaptive diagnosis and feedback. Furthermore, "...the only theory available to guide instructional development was behavior theory, which poorly matched the cognitive goals of education." (Lesgold, 1988, p. iii). Over time, researchers in AI and cognitive psychology joined forces, and together provided a basis for a new generation of computer-based teaching programs. Some of the research issues that dominated the 70's are discussed below(see 5.23).

19.4.1.1 Real-Time Problem Generation. The earliest systems to incorporate some now "classic" ITS elements were programs that generated problems and learning tasks, representing a big departure from the canned problems stored in CAI databases. For example, Uhr (1969) developed a computer-based learning system that created, in real-time, simple arithmetic problems and vocabulary recall tasks. The next major advance in this area came in the form of computer programs that generated problems that had been tailored to the knowledge and skill level of a particular student, thus providing the foundation for student modeling.

19.4.1.2. Simple Student Modeling. The Basic Instructional Program (BIP) develops procedural skills required in learning the programming language BASIC (Barr, Beard, and Atkinson, 1976). It did so by selecting problems based on what the student already knew (past performance), which skills should be taught next, and its analysis of the skills required (problems in the curriculum). Exercises were dynamically and individually selected per person (from a pool of 100 sample problems); then teaching heuristics were applied to the student model to identify skills to be taught and exercises were selected that best involved those skills. Selection of appropriate exercises was based on information contained in a network called the Curriculum Information Network (CIN), relating tasks in the curriculum to issues in the domain knowledge. Thus, a programming task in the tutor was represented in terms of its component skill requirements. Based on a task analysis, BIP knew that the component skills needed for solving a particular programming problem included such skills as: initialize numeric variable, use for-next loop with literal as final value, and so forth. Moreover, each task tapped a number of skills.

19.4.1.3. Knowledge Representation. Classic CAI used pages of text to represent knowledge, but with little psychological validity. In contrast, Carbonell's (1970) SCHOLAR program (often credited with being the first true ITS) used a semantic net to represent domain knowledge (South American geography) as well as the student model. Nodes in the network had tags to indicate whether the concept was known to the student. This novel application of semantic network as a general structure of knowledge representation supported mixed-initiative dialogs with students. Not only could the computer ask questions of the student, but the student could, theoretically, ask questions of the computer. One major limitation of this semantic knowledge representation was the difficulty of representing procedural knowledge(see 5.3).

19.4.1.4. Socratic Tutoring. Carbonell's research spawned another line of work concerned with enabling systems to engage in Socratic dialogs, believed to involve the learner more actively in the learning process. Collins (1977) outlined a set of tutorial rules for Socratic tutoring that were incorporated into a system called WHY (Stevens and Collins, 1977). For example, consider the following IF/THEN string: IF the student gives an explanation of one or more factors that are not sufficient, THEN formulate a general rule for asserting that the given factors are sufficient, and ask the student if the rule is true (Collins, 1977, pp. 343-344). Instead of semantic nets, the domain knowledge (rainfall) was stored in a "script hierarchy" containing information about stereotypical sequences of events.

19.4.1.5. Skills and Strategic Knowledge. Another attempt to stimulate thought among students (rather than being passive recipients of information) was the focus of a group of researchers at Xerox PARC in the mid- to late-1970s. For instance, WEST (Burton & Brown, 1976) was developed to help students learn/practice skills involved in the manipulation of arithmetic expressions. The goal was to move around a game board (How the West Was Won) and either advance the maximum number of squares, land on and thus "bump" an opponent back some fixed amount of squares, or take a shortcut. Not only was basic arithmetic skill involved, but also strategic knowledge was required. The system was attentive to all levels of knowledge and skill, but the "coach" was somewhat unobtrusive, sitting in the background monitoring the student's moves, intervening only when it was clear that the student was floundering. Then the coach would make a few suggestions to enhance student skills. WEST's coaching goals were accomplished by focusing on the strategy used to construct a move (viz.,"issue-based" tutoring).

19.4.1.6. Reactive Learning Environments. In reactive learning environments, the system responds to learners' actions in a variety of ways that extend understanding and help change entrenched belief structures using examples that challenge the learner's current hypotheses. An early, excellent example of this kind of environment was SOPHIE (Sophisticated Instructional Environment), designed to assist learners in developing electronic troubleshooting skills (see Brown & Burton, 1975; Brown, Burton, & deKleer, 1982). For instance, in SOPHIE I, learners located faults in a broken piece of equipment. They could ask SOPHIE questions in English (e.g., to obtain values of various measurements taken on the device). SOPHIE I included three main components: a mathematical simulation, a program to understand a subset of natural language, and routines to set up contexts, keep history lists, and so on. A student, troubleshooting a simulated piece of equipment, could offer a hypothesis about what was wrong. SOPHIE I reacted to the request by comparing the hypothesis to the measurements entered by the student. SOPHIE II extended the environment of its predecessor by adding an articulate expert based on a pre-stored decision tree for troubleshooting the power supply that was annotated with schema for producing explanations. SOPHIE III represented a significant advance; it contained an underlying expert based on a causal model rather than on a mathematical simulation. The importance of this change is that, in SOPHIE I, the simulator worked out a set of equations not using human-like, causal reasoning, so it wasn't possible for the system to explain its decision in any detail. But SOPHIE III did employ a causal model of circuits to deal with the student feedback deficiency. Research with SOPHIE spawned a lot of later research in troubleshooting, reactive learning environments, and articulate experts.

19.4.1.7. Buggy Library. Brown and Burton (1978) also developed BUGGY, a frequently cited example of a system employing a "buggy" library approach to the diagnosis of student errors. BUGGY was a framework for modeling misconceptions underlying procedural errors in addition and subtraction where students' errors were represented as the results of "bugs" (errors) in an otherwise correct set of procedures. DEBUGGY (Burton, 1982) was developed as an off-line version of the system based on the BUGGY framework using the pattern of errors from a set of problems to construct an hypothesis concerning a bug, or combination of bugs, from the library that generated the errors. IDEBUGGY (Burton, 1982) was an on-line version of BUGGY, diagnosing the student's procedure bit-by-bit while giving the learner new problems to solve. The major limitation of these kinds of systems was the inability to anticipate all possible misconceptions. Moreover, bugs could appear transient as they were being repaired.

19.4.1.8. Expert Systems and Tutors. MYCIN (Shortliffe, 1976) was a rule-based expert system for diagnosing certain infectious diseases such as meningitis. GUIDON (Clancey, 1979) was constructed to interface with MYCIN for tutoring, interactively presenting the rules in the knowledge base to a student. This tutoring operated as follows. GUIDON described case dialogs of a sick patient to the student in general terms. The student had to adopt the role of a physician and ask for information that might be relevant to the case. GUIDON compared the student's questions to those which MYCIN would have asked and then responded accordingly.

19.4.1.9.Overlay Models/Genetic Graph. The definition of an overlay model is one of a novice-expert difference model representing missing conceptions. It's typically implemented as either an expert model annotated for missing items, or an expert model with weights assigned to each element in the expert knowledge base. To illustrate how it works, consider WUSOR (Stansfield, Carr, & Goldstein, 1976)--the name of the on-line coach for the game WUMPUS (Yob, 1975). The WUMPUS player had to traverse through successive caves to locate the hiding Wumpus. Many dangers faced the player (e.g., pits, bats), but the problem could be solved by applying logical and probabilistic reasoning to information obtained along the way. The goal of the game was to shoot an arrow into the Wumpus' hiding spot before you were killed. WUSOR evolved through (at least) three generations, each with a progressively more sophisticated student model. The first version had only an expert and advisor and did not try to diagnose the learner's state of knowledge. The next version (II) incorporated an overlay model (Carr & Goldstein, 1977) where the expertise was represented as rules, and the student's knowledge state was a subset of the expert's knowledge. Goldstein (1979) made the final transformation to WUSOR (III) by including the genetic graph, combining overlay modeling (rule-based representation) with a learner-oriented set of links between curricular elements. "Genetic" related to the notion of knowledge being evolutionary, and graph denoted the relationships between parts of knowledge expressed as links in a network. A genetic graph could represent type-of-links (e.g., generalization, analogy, refinement) as well as deviation links (i.e., buggy rules as opposed to simply absent ones).

The 1970s were marked by experimental systems that bore little resemblance to one another. During the following decade, systems became less idiosyncratic, but there was still a lot of diversity in the field.

19.4.2 1980s: Standardized Approaches and Environments

The 1980s were characterized by enormous growth and momentum in the ITS field. By the mid-1980s, the development of tutors greatly exceeded their evaluations; everyone wanted to participate in the excitement of building ITS, but few cared to test their system's efficacy (Baker, 1990; Littman & Soloway, 1988; see 12.2, 39.4). Sleeman (1984) attempted to focus research efforts by outlining four main problems with ITS at the time:

1. Feedback specificity--Instructional feedback was often not sufficiently detailed for a particular learner.
2. Non-adaptability--Systems forced students into their own conceptual framework rather than adapting to a particular student's conceptualization.
3. Atheoretical foundation--Tutoring strategies used by the systems lacked a theoretical cognitive foundation.
4. Restrictive environment--User interaction and exploration was too restricted.

These main criticisms were addressed, to varying degrees, during the 1980s.

19.4.2.1. Model tracing. Anderson and his colleagues at Carnegie-Mellon University developed a model-tracing approach to tutoring based on production systems as a way of modeling student behavior. The model-tracing approach has been employed in a variety of tutoring systems, such as the LISP tutor (Anderson, Boyle & Reiser, 1985) and the Geometry tutor (Anderson, Boyle & Yost, 1985). Model tracing provides a powerful way to both validate cognitive theories (e.g., Anderson, 1987) and to deliver low-level, personalized remediation. The approach works by delineating many hundreds of production rules that model curricular "chunks" of cognitive skill. A learner's acquisition of these chunks is monitored (i.e., the student model is traced), and departure from the optimal route is immediately remediated.

In theory (and practice) the model-tracing approach for the Geometry and LISP tutors is so complete that it captures an enormous percentage of all students' errors. A major drawback is that this approach does not allow students to commit those errors themselves. As soon as there is a mis-step, the tutor cries "foul" and stops the student from doing anything else until the correct step is taken. As Reiser points out (e.g., Reiser, Ranney, Lovett & Kimberg, 1989), the student is not only prevented from following these mistakes to their logical conclusion (and getting hopelessly confused) but also prevented from obtaining an insight into the mistake (i.e., that the mistake is obvious). These are some of the best learning experiences students can have, but they appear to be blocked by the model-tracing approach.

Model tracing challenges the first criticism (feedback specificity). That is, the grain-size of feedback is as small as you can get (i.e., the production level) thus providing the most detailed, specific feedback possible. However, in some cases (i.e., for certain students or particular problems), this level of feedback may be too elemental, the forest is lost for the trees. Next, as mentioned above, the systems can adapt to a wide range of student conceptualizations, challenging the second (non-adaptability) criticism. The approach also demolishes the third criticism (atheoretical foundation), as it was explicitly based on Anderson's cognitive theory (ACT*). The positive features of this approach, however, are achieved at the expense of the fourth (restrictive environment) criticism. That is, the model-tracing approach is restrictive. To accomplish the necessary low-level monitoring and remediation of this approach, the learner's freedom has to be curtailed. So, learning by one's mistakes is out (which is often a powerful way to learn). A final drawback of this approach is that, while it works very well in modeling procedural skill acquisition, it does not work well for domains that are ill-structured, or that are not rule-based (e.g., Creative writing, Economics, Russian history).

19.4.2.2. More Buggy-Based Systems. During this time period, a plethora of tutors was developed based on the "buggy" library approach (see BUGGY, above). While these systems do provide very specific feedback about the nature of the learner's error (countering criticism 1, feedback specificity), the system response is dependent on the program's ability to match the student's error with that of a stored "bug." Along these same lines, as with model tracing (because only stored bugs are acknowledged), novel bugs are ignored; thus there is no way to update the buggy library or adapt to the learner's current conceptualization (criticism 2, non-adaptability). This approach is theoretically based on the notion of cognitive errors in specific procedures, impasse learning, and repair theory (VanLehn, 1990), countering criticism 3 (atheoretical foundation). Finally, these systems constrain the learner somewhat less than the model-tracing approach; thus, it is a response to criticism 4 (restrictive environment).

A good illustration of a system based on the buggy approach is PROUST (Johnson, 1986; Littman & Soloway, 1988), designed to diagnose nonsyntactic student errors in Pascal programs. The system works by locating errors in students' programs where they compute various descriptive statistics such as the minimum and maximum values, and averages. The major drawback of this system is that it is implemented off-line. In other words, the tutor has access to a final product on which to base its diagnosis of student errors--completed student programs are submitted to PROUST, which prints out the diagnosis (Johnson & Soloway, 1984).

A parallel "buggy" research project involved a system called PIXIE (Sleeman, 1987), an on-line ITS based on the Leeds Modeling System (LMS), a diagnostic model for determining sources of error in algebra problem solving due to incorrect procedural rules or "mal-rules." While some may equate mal-rules with buggy rules, they differ in a fundamental way. Sleeman created them by postulating a set of basic buggy rules from which higher order mal-rules could be generated from the structure of the knowledge base itself. Mal-rules are inferred from basic principles and bugs; they are at a level of abstraction above bugs. In fact, John Anderson makes the same point about his model-tracing procedures. Because of the complexity of his model-tracing productions, many productions fire or are used over and over again in contexts for which they were not first generated, and so they too take on a kind of abstract or general quality in his framework.

The major problem with LMS is that it only diagnoses the incorrect rules; it does not remediate.

19.4.2.3. Case-Based Reasoning. Another category of systems emerging at this time came from case-based reasoning (CBR) research (Schank, 1982; Kolodner, 1988). Proponents of this approach suggest that the goal of ITS should be to teach cases and how to index them. Given that the student, not the program, is the one doing the indexing, this system affords the learner greater freedom, and promotes a more adaptive learning environment (countering criticisms 4--restrictive environment and 2--non-adaptability, respectively). Furthermore, whereas the model-tracing tutors work poorly in ill-structured domains, CBR works well in those areas (e.g., politics, philosophy). This tradeoff, however, can result in less specific feedback to learners (criticism 1, feedback specificity).

These CBR systems also perform well in domains where there are too many rules, or too many ways in which rules can be applied (e.g., programming, game playing). CBR suggests approximate answers to complex problems, thereby limiting how many rule combinations should be explored. There are two main processes involved with CBR: indexing (labeling new experiences for future retrieval) and adaptation (changing a retrieved case to fit a current situation). Further, two kinds of indices are required: concrete and abstract. Concrete indices refer to objects and actions usually directly mentioned in the case, while abstract indices refer to more general characterizations. The "indexing problem" deals with ways to determine the correct abstract and concrete indices for cases. How one indexes new cases determines what cases one will compare the inputs against. Using a general index, one can retrieve a case even when it shares no specific details with the current situation.

Schank has made some very provocative statements about the human mind as a story teller, and about the need to encapsulate knowledge into stories, not into hierarchical data structures like semantic networks. But his procedures have yet to lead to any of the other strong characteristics of ITS that we emphasize in this paper: student models, teaching models, bugs, and so on. Instead, they exist as very generative and interesting systems. As such, they have something in common with microworlds; that is, people enjoy exploring them and can learn from them, particularly those regarding ill-structured and complex domains. However, when students don't learn, or manifest some misconception(s), the very same looseness of structure and organization in these systems prevents them from determining why, and doing something about it. Finally, according to Riesbeck and Schank (1990), case-based reasoning (CBR) serves as a model of cognition and learning. But, while these systems present a provocative and well-conceived approach that has many practical and obvious merits, they cannot be said to possess a solid theoretical foundation (criticism 3, atheoretical foundation).

A major limitation of this approach includes the problem of anticipating and representing a sufficient number of cases to be cataloged.

19.4.2.4. Discovery Worlds. With just a few exceptions, learning from computers in the 1960s and 1970s was characterized by inflexible presentations of didactic material. But an opposition movement arose in the 1970s that gained steam in the 1980s; it resulted in the development of discovery learning environments(see 7.4.1). These computerized systems (typically a computer simulation environment with simple interface and tools) were designed to make it possible for students to acquire various knowledge and skills on their own. For example, students could learn LOGO (Papert, 1980; see 12.3.2, 24.5) or Newton's laws of motion (White, 1984) within discovery (or micro) worlds. Typically, feedback was "natural" or implicit, not specifically explained to the learner (relating to criticism 1, feedback specificity).

One of the main strengths of these systems was their great adaptability to a range of different learners (countering criticism 2, non-adaptability). Students were free to explore and act within the microworld as they chose; with the ramifications of their actions immediately revealed, countering criticism 4 (restrictive environment). This movement was based on the theoretical premise that in discovery learning, one can radically alter the perceptual relationship between the learner and the knowledge or skills to be acquired, thus addressing criticism 3 (atheoretical foundation). This position was epitomized by Piaget (1954) who stated that, "...an education which is an active discovery of reality is superior to one that consists merely in providing the young with ready-made wills to will with, and ready-made truths to know with."

A major drawback of these systems is that not all persons are skilled in the requisite inquiry behaviors necessary to achieve success in these environments (see Shute & Glaser, 1990). That is, to be successful, an individual should be able to: formulate efficient experiments, state, confirm, and/or negate hypotheses; appropriately relate hypotheses and experiments; plan future experiments and tests; engage in self-monitoring, and so on.

19.4.2.5. Progression of Mental Models. White, Frederiksen and their colleagues (Frederiksen, White, Collins, & Eggan, 1988; White & Frederiksen, 1987; White & Horowitz, 1987) incorporated ideas from (a) AI research on mental models and (b) qualitative reasoning to develop QUEST (Qualitative Understanding of Electrical System Troubleshooting) as well as "Thinker Tools." This approach, like model-tracing, above, is thus theoretically grounded (in opposition to criticism 3, atheoretical foundation).

These systems work by motivating students to want to learn by pointing out errors and inconsistencies in their current beliefs. Then students are guided through a series of microworlds, each more complex than the one preceding, toward the objective of more precise mental models of the evolving subject matter (e.g., electrical concepts or Newtonian mechanics). Finally, students formalize their developing mental models(see 5.3.7)by evaluating a set of laws describing phenomena in the microworld; then they apply the selected law to see how well it predicts real-world events.

These systems promote learning, neither completely free nor overly restricted (relating to criticism 4, restrictive environment), that resides about halfway between true discovery environments and model-tracing environments. A programmed series of mental models produces higher-level feedback compared to, for example, feedback at the production level (addressing criticism 1, feedback specificity). Finally, the systems can adapt to a wide range of learner misconceptions (challenging criticism 2, non-adaptability).

19.4.2.6. Simulations. Graphical simulations have become more central to the ITS enterprise as the power of computers has grown. Along with increasing computational power, software systems have grown more complex; object-oriented systems can now mimic devices of great complexity and interactivity. Simulations are useful wherever real objects are involved in a learning or training task, and they provide many benefits over real devices. Not only are they less dangerous, less messy, and exactly replicable; simulations are inspectable and self-explanatory in ways that real objects cannot be. Simulations not only display aggregate behavior, but they are decomposable into constituents that mimic novice or expert mental models. This decomposability of graphic displays and simulations mimics the power of productions in expert systems for creating natural chunks that promote learning(see 17.4).

Early ITS, like SOPHIE, could generate only very simple line drawings. A dramatic increase in the power of graphic simulations took place with Steamer (Hollan, Hutchins & Weitzman, 1984) and the use of personal LISP machines. These machines could generate interactive graphics with animated components. It was not long before this graphical power became available for ITS on smaller personal computers that could be used in industrial and educational settings. Of course, more powerful systems that were developed in the 1980s, like Hawk MACH-III, could expand the number of components and complexity of the animations by orders of magnitude (Kurland, Granville, & MacLaughlin, 1992). Using object-oriented constructions, MACH-III made each part of complex radar systems inspectable and self-explanatory. For teaching troubleshooting, each decomposable part of the radar device could even explain its role in the troubleshooting sequence for any fault that had been created in the system. Given this power and complexity, these systems were stretched to their limits and brought to their knees by additional requirements for student models, curriculum sequences, and hypertext interfaces. Even though these computer simulations were forced to operate at the edge of their acceptability, an official Army evaluation verified the many benefits of simulation-based training systems (Farr and Psotka, 1992).

Depending on the level that a simulated device has been decomposed to, and the degree of learner response regarding manipulations and ensuing ramifications, feedback could attain various levels of detail (criticism 1, feedback specificity). Furthermore, as simulations become typically very reactive to learner actions, they can serve as a direct challenge to the second criticism (non-adaptability). Simulations, similar to discovery worlds, also leave quite a bit of freedom to explore and manipulate simulated objects and devices (countering criticism 4, restriction environments). However, the drawback of these systems is that a solid theoretical basis is lacking (criticism 3, atheoretical foundation). Simulation research in the 1980s spurred later work that attempted to incorporate pedagogical strategies into the simulation-based systems. Moreover, related developments continue to evolve in complexity with the addition of Virtual Reality interfaces to three dimensional models and simulations (Acchione-Noel and Psotka, 1993).

Two other areas of research and development gained prominence at this time: Natural language processing (NLP) and authoring shells. While these research spheres were important in relation to ITS research, they could be applied within a variety of tutor types. For example, NLP could be used to communicate information to the learner (or accept input from the learner) in model-tracing tutors, discovery worlds, and so forth. And authoring shells could be built for the development of a range of tutoring systems. Because of this openness, the following two ITS-related issues won't be discussed in relation to our four criticisms, listed earlier.

19.4.2.7. Natural Language Processing (NLP). This technology was an important part of ITS right from the beginning. SOPHIE, in fact, was built on a powerful and original NLP technique developed by Richard Burton; it was called Semantic Grammar. Representing a powerful combination of carefully selected keywords with algorithms that searched the context for meaningful variables and objects, it worked surprisingly well, given its relative simplicity. Since communication is such an important element of ITS (see Wenger, 1987 for emphasis), it is not surprising that NLP technologies have been used in several ITS for discourse networks (Woolf, 1988) and especially for language instruction (Yazdani, 1990; Psotka, Holland, & Kerst, 1992). The development of powerful, efficient Prolog compilers and languages on PCs has led to the implementation of some interesting instructional grammars that can handle discourse in English or other languages, and provide multimedia instruction in advanced language concepts and grammar, as well as simple vocabulary and verb declension. The potential addition of animations and immersion into Virtual Environments adds a bright new prospect to the old goal of immersive language learning.

19.4.2.8. Authoring Systems. The creation of computer-based environments to facilitate the design and development of ITS has been an important and continuing thread of research. The goal of authoring systems is to give relative computer novices a software toolkit to take advantage of the power of computers for designing instruction. An example of one powerful graphic authoring system developed over the last decade is that by Towne and Munro (1992).

Quite powerful CBT systems have been made available over the years. Research, beginning in the 1980s, attempted to adapt such systems as authoring shells for developing ITS. Miller and Lucado (1992) were among the first to integrate the power of CBT authoring environments with the technology of ITS. Their prototype system was the harbinger of many more powerful combinations of traditional CBT and next generation ITS technologies. Most recently, DARPA has funded a unique consortium of Apple Computer, textbook publishers such as Houghton-Mifflin, and ITS experts Beverly Woolf and John Anderson to begin the development of next-generation authoring tools for instruction and training.

The relative quiescence of the 80s transitioned into the current state of ITS affairs, marked by a perception of instability and controversy.

19.4.3 1990s: Great Debates

The four hot ITS topics right now may be broadly characterized as: (a) How much learner control should be allowed in systems? (b) Should learners interact with ITS individually or collaboratively? (c) Is learning situated, unique, and ongoing, or symbolic and does it follow an information-processing model? and (d) Does virtual reality (VR) uniquely contribute to learning beyond CAI, ITS, or even multi-media? There are, of course, proponents and opponents to each of these positions.

19.4.3.1. Degree of Learner Control. The debate over the amount of learner control that should be a part of the learning process has raged for many years(see 7.4.6, 12.2.3.5, 14.6.2, 22.5.5, 23.7.2, Chapter 33). On the one hand, some have argued that discovering information on one's own is the best way to learn (e.g., Bruner, 1961). On the other hand, structure and direction have been stressed as the important ingredients in the promotion of student learning (e.g., Ausubel, 1963). The same debate has appeared in the ITS arena. Two differing perspectives, representing the ends of this continuum, have arisen in response to the issue of the optimal ITS learning environment. One approach is to develop a computerized environment containing assorted tools, and allow learners freedom to explore and learn independently (e.g., Collins & Brown, 1988; Shute, Glaser, & Raghavan, 1989; White & Horowitz, 1987). Advocates of the opposing position argue that it is more effective to develop straightforward learning environments with no digressions permitted (e.g., Anderson, Boyle & Reiser, 1985; Corbett and Anderson, 1989; Sleeman, Kelly, Martinak, Ward, & Moore, 1989). This disparity between perspectives becomes more complicated because the issue is not just which is the better learning environment, but which is the better environment for whom, a classic aptitude-treatment interaction question (see 22.3.3; Cronbach and Snow, 1981). There are, undoubtedly, temporal aspects to this issue as well. For instance, it may be more efficient to learn a new cognitive skill initially by direct instruction, then later, by greater exploration. In this way, learners can better control their own learning process.

Merrill, Reiser, Ranney, and Trafton (1992) investigated how human tutors dealt with the issue of learner control. They compared human- to computer-tutoring techniques, and found that, while expert human tutors did sometimes act like model-tracers, they actually maintained a "delicate balance" between (a) allowing students freedom and control and (b) giving students sufficient guidance. In general, pedagogical research findings differ with regard to the amount of learner control to allow in automated systems (e.g., Fox, 1991; Lepper, Aspinwall, Mumme, & Chabay, 1990; Merrill, Reiser, & Landes, 1992). In addition to the temporal factor cited above, this issue of learner control is also greatly dependent on other variables, such as the subject matter being instructed, the desired knowledge or skill outcome, incoming aptitudes, and so on (see Kyllonen & Shute, 1989, for a complete discussion of these interacting variables). That is, if the desired learning outcome is a smoothly executed skill, it may be more efficient to instruct certain learning tasks with direct instruction and plenty of practice. But if the desired learning outcome is a functional mental model of relevant principles, an exploratory environment, complete with various components such as on-line circuits, ammeters, and resistors, may be what is needed to achieve that educational objective.

Most current computer-administered instructional systems do not foster self-reliance in students, or encourage them to seek new information on their own. To rectify this deficit, Barnard, Erkens, and Sandberg (1990) propound the building of more flexible systems packaging communication expertise as a separate component. With less learner initiative, it's much easier to interpret input, but at what cost to learning outcome? In Japan, research is being conducted along these lines. The concept and development of ITS is becoming merged with interactive learning environments (ILE) to produce what is referred to as a "bi-modus learning environment" (BLE) (Otsuki, 1993). Whereas the main strength of ITS is its ability to derive a student model based on the identification of acquired rules, its main weakness is the inability to help learners acquire new knowledge by themselves. In contrast, students in an ILE can extract and comprehend rules induced from a complex domain, but the ILE cannot explicitly identify a student's misconceptions or tutor them in terms of their comprehension level. Thus the two (ITS and ILE ) are complementary to one another, and BLE represents combining the strengths of each.

Another way to increase learner control has been suggested by Bull, Pain, and Brna (1993). Their intriguing alternative to traditional student modeling, that of replacing the burden of the ITS, is to produce accurate representations of the learner's knowledge state; the learner is empowered with greater control, e.g., to construct and repair the model. Bull and associates contend that their model will result in a more accurate representation of the learner's beliefs, and thus be more highly regarded by the student. The learner is expected to benefit through the reflection necessary to accomplish this modeling task. Unfortunately, no data have yet been collected about the efficacy of this novel approach.

"Coached practice environments" (i.e., Sherlock I and II) represent yet another way to provide control during learning by combining apprenticeship training with intelligent instructional systems (Lajoie & Lesgold, 1992; Lesgold, Eggan, Katz, & Rao, 1992). These systems support greater learner initiative because the apprentice learns by doing (singularly or collaboratively); knowledge is anchored in experience; and the coach provides knowledge within an applicable context. Intelligent systems are developed with many of the characteristics of human apprenticeships, and performance can be easily assessed. Through replay and comparisons with the expert performance, this approach also supports trainee analysis of performance.

Salomon (1993) supports the trend of moving away from building traditional ITS and towards the design of systems as cognitive tools. He sees cognitive tools manipulated by students as instruments that promote constructive thinking, transcending cognitive limitations, and making it possible for students to engage in cognitive operations they wouldn't otherwise have been capable of. Some ITS programs make most diagnostic and tutorial decisions for the student; therefore they are not really cognitive tools because, "they are not designed to upgrade students' intelligent engagements." (p. 180). Also in accordance with the notion of computers as learning tools, learners should have the option to alter the degree of control themselves, from none (e.g., didactic environment) to maximum (e.g., discovery environment), as necessary.

By shifting toward increased learner control, are individuals who are not very active or exploratory by nature being penalized or handicapped? Shute and Glaser (1990) investigated individual differences in learning from a discovery environment (Smithtown) and found that individuals who demonstrated systematic, exploratory behaviors (e.g., recording baseline data, limiting the number of changed variables) were significantly more successful in Smithtown compared to those who revealed less systematic behaviors. On the basis of that finding, they hypothesized in a different study (using an electricity tutor) that high-exploratory individuals would learn more from an inductive environment (than from a more directed, applied environment), and less-exploratory learners would benefit from a supportive, applied environment (compared to an inductive one). A person's exploratory level was quantified based on certain indices (e.g., number of tries and length of time spent changing a resistor value, using the on-line voltmeter or ammeter). Subjects were randomly assigned to one of two learning environments, and the data were analyzed, post hoc. The hypothesized learning style by aptitude interaction was supported by the data (Shute, 1993-b). So, discovery learning environments do not suit everyone equally well. For some, they provide a really bad fit. To determine whether this kind of learner style by treatment interaction is replicable, Shute (1994) conducted a confirmatory test of the same ATI, reported above. Subjects were placed a priori in one of two environments based on the decision rule obtained from the previous study. And, in fact, the ATI was confirmed(see 11.4.4, 22.3.6, and 33.6 for more on ATIs).

In conclusion, a mid-point between too much and too little learner control is probably the best bet as far as optimal ITS learning environment. Furthermore, this milestone should not be fixed, but should change in response to learners' evolving needs. Finally, learners should have some input into the design of the environment, as well.

Our next debate addresses the issue of whether learning alone is better or worse than learning in conjunction with others (where "others" may mean other humans, or with a computer acting as a "partner" in the learning process). As with everything relating to learning, there is probably no clear cut answer to this question; there is no "overall" superior way to learn. Rather, it is almost certain that interactions exist, where solo learning may be superior for certain topics (e.g., rote memorization of multiplication tables) or for particular learner types (e.g., highly motivated individuals). Collaborative learning may be more effective for other domains or persons. While we don't specifically address these interactions in the following discussions, they should be kept in mind(see refer to Chapter 6, 7.4.8, 23.4.4, Chapter 35).

19.4.3.2. Individual vs. Collaborative Learning. Traditionally, ITS have been designed as single-learner enterprises. Bloom (1984) and others have presented compelling evidence that individualized tutoring (using human tutors) engenders the most effective and efficient learning across an array of domains (see also Shute & Regian, 1990; Woolf, 1988). Furthermore, intelligent tutoring systems epitomize this principle of individualized instruction. In his often-cited 1984 paper, Bloom presented a challenge to instructional researchers that has been called the "two sigma problem." The goal is to achieve two standard deviation improvements with tutoring over traditional instruction methods. So far, this goal has yet to be attained using individualized ITS.

An alternative approach to individualized instruction is collaborative learning, the notion that students, working together, can learn more than by themselves, especially when they bring complementary, rather than identical, contributions to the joint enterprise (Cummings & Self, 1989). Collaboration is defined as a process by which "individuals negotiate and share meanings relevant to the problem-solving task at hand." (Teasley & Roschelle, 1993, p. 229), and is distinct from cooperation which relates to the division of labor required to achieve some task.

Two empirical questions relevant to this chapter include: (a) Are two heads better than one? and (b) Can intelligent computer systems support collaborative learning endeavors? Recently, research is beginning to shed light on both of these questions. For example, many researchers have shown impressive student gains in knowledge and skill acquisition from collaborative learning environments (e.g., Brown & Palincsar, 1989; Lampert, 1986; Palincsar & Brown, 1984; Scardamalia, Bereiter, McLean, Swallow, & Woodruff, 1989; Schoenfeld, 1985). Furthermore, the few studies of the effectiveness of collaborative learning in computer-based learning environments have also been positive (e.g., Justen, Waldrop, & Adams, 1990; Katz & Lesgold, 1993; Papert, 1980).

There are basically two ways of implementing collaborative learning environments using computers: (a) a small group of learners interact with a single intelligent computer system, or (b) the computer system itself serves as the "partner" in the collaboration. The first way (i.e., a small group using one computer) represents an extension of the research on collaborative learning in classrooms. In this case, some of the issues that need to be addressed have been outlined by Teasley and Roschelle (1993). The system must be able to: (a) introduce and accept knowledge into a joint problem-solving space, (b) monitor ongoing activities for evidence of divergences in meaning, and (c) repair divergences that impede the progress of the collaboration. The difference between this list and general modeling issues in ITS is that it deals with a student model that's built upon a joint, rather than single, problem solving space. The second way of implementing collaboration (i.e., assigning the computer as the learner's partner) represents an intriguing twist on the notion of collaborative learning. To illustrate, Cummings and Self (1989) proposed a collaborative intelligent education system (IES) that engages the learner in a partnership. Here, the computer serves as a collaborator, not as an authoritarian instructor. In both cases, a student model still must be derived, either that of an individual or a group.

Additional research and controlled studies must be conducted in order to test the relative efficacy of collaborative versus individualized instruction. For a variety of reasons (e.g., greater range of shared knowledge, resource limitations, etc.), the notion of collaborative learning environments is appealing. There are a lot of unanswered research questions that need to be addressed, however. Some of these (listed in Katz & Lesgold, 1993) include: What parts of the curriculum should be learned collaboratively, and what parts learned individually? What teaching methods should be used to achieve the instructional goals, and how should they be sequenced to optimize learning? What should the computer tutor do while students work on problems? What additional roles could the computer coach perform? This area of research is also likely to shed light on the interactions mentioned earlier. We now present the third hot topic, namely, the nature of learning and its impact on ITS design.

19.4.3.3. Situated Learning Controversy. To supporters, this is not just a trend, but a radically new perspective (or philosophy) that supports the integration of "...psychological theories of physical and cognitive skills, uniting emotions, reasoning, and development, in a neurobiologically grounded way." (Clancey, 1993, p. 98). It has also been referred to in the literature as "situated action" and "situated cognition." Recently, several prominent journals have devoted entire issues to the debate concerning the value of situated learning compared to the more standard paradigms (e.g., ACT*, SOAR): 1993 Cognitive Science, 17(1), and 1993 Journal of Artificial Intelligence and Education, 4(1)(see 12.3.1.2, 8.7, 8.9).

Obviously, one's belief in either situated cognition or the traditional information-processing model has implications for the design of ITS. To illustrate this distinction, first consider Greeno's summary of situated cognition's perspective on where knowledge resides: "Rather than thinking that knowledge is in the minds of individuals, we could alternatively think of knowledge as the potential for situated activity. On this view, knowledge would be understood as a relation between an individual and a social or physical situation, rather than as a property of an individual." (Greeno, 1989, p. 286). Next, consider the nature of knowledge from the information-processing perspective. Anderson's (1983) ACT* theory proposed two fundamental forms of knowledge: procedural, represented in the form of a production system, and declarative, represented in the form of a node-link network of propositions(see 5.4, 29.2). Both representations are believed to operate within long-term and short-term memory structures.

These two positions present quite different views on how learning, or knowledge acquisition, occurs. In the first case (situated cognition), learning is a process of creating representations, inventing languages, and formulating models for the first time. Learning is ongoing, occurring with every thought, perception, and action, and is situated in each unique circumstance. Situated cognition argues for an instructional system rich with explicit tools and varied exemplars that can support and extend learners' discovery processes. "Insight is more likely when the problematic situation is so arranged that all necessary aspects are open to observation." (Bower & Hilgard, 1981, p. 319).

The second position (information processing) sees learning as progressing from declarative knowledge, to procedural skills, to automatic skills, dependent upon: enablers (i.e., what one already knows and can transfer to new situations) and mediators (i.e., cognitive processes determining what one can acquire, such as working-memory capacity and information processing speed) (e.g., Anderson, 1983, 1987; Kyllonen & Christal, 1990). Thus, learning refers to the addition and restructuring of information to a database, in accordance with specific learning mechanisms (e.g., knowledge compilation, transfer). To facilitate learning, one must build a system that can (a) Analyze the initial state of knowledge and skill; (b) Describe the desired or end state of knowledge and skill (learning outcome); and (c) Present material and problems that will transition a learner from initial to desired state. This kind of tutoring system is based on a well-defined curriculum that's been so arranged to promote knowledge/skill acquisition (or facilitate transition from current to goal state).

It may be that these two positions are mutually exclusive. That is, knowledge either resides internally in one's head, or externally, in the environment. Alternatively, it may be that there is some overlap, whereby some forms of knowledge are stored, and some derivable from the current situation. In a preliminary attempt to bridge the gap between situated- and traditional-learning models, Shute, Gawlick-Grendell, and Young (1993) have recently developed a series of statistics modules, Stat Lady. Learning is situated within various gaming environments (e.g., "Stat Craps"); the theoretical postulates are that learning is a constructive process, enhanced by experiential involvement with the subject matter, that is situated in real-world examples and problems. Furthermore, the system has a well-defined curriculum in accordance with popular learning theory.

According to constructivism, learners actively construct new knowledge and skills, either from what they already know (information-processing premise) or from what resides in the environment (situated cognition stance). Both positions would probably agree that learners do not come to a learning situation with a tabula rasa, but rather, as active-pursuers (not passive-recipients) of new knowledge (e.g., Bartlett, 1932; Collins, Brown, & Newman, 1989; Drescher, 1991; Edelman, 1987; Piaget, 1954). Both positions also support the position that the construction process can be enhanced by environments supporting experiential learning. Research in this area has shown that knowledge derived experientially tends to be more memorable than passively-received knowledge because the experience ("doing" rather than "receiving") provides cognitive structure, and is intrinsically motivating and involving (e.g., Friedman & Yarbrough, 1985; Harel, 1991; Harel & Papert, 1991; Shute & Glaser, 1991; Spencer & Van Eynde, 1986). Finally, when instruction is situated (or anchored) in interesting and real-world problem-solving scenarios, that also is believed to enhance learning (Brooks, 1991; Brown, Collins, & Duguid, 1989; Clancey, 1992; Collins, Brown, & Newman, 1989; Lave & Wenger, 1991; Suchman, 1987; The Cognition & Technology Group at Vanderbilt, 1992).

The Cognition and Technology Group at Vanderbilt (1992) has also been working on developing a pedagogical approach to situated cognition. They define "anchored instruction" as an attempt to actively engage learners in the learning process by situating instruction in interesting and real-world problem-solving environments. Rather than teaching students how to solve particular problems, these systems teach generalizable skills, helpful across a variety of problem-solving situations. The major goal of this type of instruction is to create authentic-feeling environments in which one can explore and understand problems and opportunities experienced by experts in a domain, and learn about the tools these experts use. This group has developed a series of adventures for middle-school students focusing on math problem formulation and problem solving. These are the "Adventures of Jasper Woodbury" series. The goal of the project is to facilitate broad transfer to other domains, embodying several design principles: (1) video-based presentation, (2) narrative format, (3) generative learning, (4) embedded data design, (5) problem complexity, (6) pairs of related adventures, and (7) links across the curriculum.

One of the major problems with this whole debate over situated cognition versus traditional information processing models is that the former position simply has not tested its underlying hypotheses at this time, while the latter has enjoyed decades of solid research. Vera and Simon (1993), rebutting Clancey's support paper(s) for situated learning, stated, "Clancey leaves us with philosophy (whether correct or not is another matter), but with precious little science." (p. 118). And that appears to be true. Because cognitive psychology is an empirical science, studies need to be conducted that examine claims made by any new position(see 42.5.2). For instance, supporters of our final "hot topic" of the 90's (Virtual Reality, or VR) claim that this new technology can improve learning by virtue of fully immersing the learner in the learning process (learning by saturation). But is there any veracity to this claim? It is certainly testable. The relationship between experience, learning, and pedagogy is a briar patch of thorny questions. Recent theoretical harangues on the nature of situated learning have laid a kind of groundwork for VR by arguing for an epistemology of learning based on experience.

19.4.3.4. Virtual Reality and Learning. A collection of technologies, known as Virtual Reality (VR), has recently been exciting the instructional technology community. This new technology refers collectively to the hardware, software, and interface technologies available to the user interested in experiencing certain aspects of a simulated 3-dimensional environment. The simulated aspects of the environment ("world") currently include a stereoscopic, low-to-medium fidelity visual representation displayed on a head-mounted display system. Using head-tracking technologies, one can update the display in accordance with head and body motions. This feature, along with the stereo disparity of the images on the two screens (one for each eye), support the illusion of moving around in 3-dimensional space(for more, see Chapter 15).

Unquestionably, VR changes the relationships between learning and experience, highlighting the role of perception (particularly visual), in learning. Experience is both social and perceptual, and VR epitomizes the notion of experiential learning. Many systems are now being developed that have demonstrated the success of the experiential approach. The current question is: Does VR represent the next logical, developmental step in the design of instructional systems? In other words, does the immersion experience (i.e., extra fidelity and related cost) significantly improve learning and performance beyond the more traditional pedagogical approaches?

Recently, there have been some empirical data collected on the relative success of VR in terms of instructional effectiveness, as well as skill transfer to the real world. For instance, Regian, Shebilske, and Monk (1992) showed that people can, indeed, learn to perform certain tasks from virtual environments (e.g., console operations, and large-scale spatial navigation). Next, knowledge and skill acquired in a VR have been shown to transfer to performance in the real world. Regian, Shebilske, and Monk (1993) found that: (a) VR console operations training can transfer/facilitate real world console operations performance, and (b) VR spatial navigation training successfully transfers to real-world spatial navigation. In contrast to the Regian, et al. (1993) findings, however, those reported by Kozak, Hancock, Arthur and Chrysler (1993) showed no evidence for transfer of a "pick and place" task from VR to the real world. However, the criterion task used in that study was quite easy; thus, the conclusions may actually be inconclusive. So, even with the relatively poor fidelity and interface currently available in VR technology, there is some evidence for its efficacy and potential as a serious learning/training environment(see 15.8).

Another positive example of VR's potential for training was presented by Psotka (1993) who argued that VR creates one uniform point of view on any representation that overcomes the conflicts and cognitive load of maintaining two disparate points of view (Sweller, 1988). The reduced cognitive overhead resulting from the single "egocenter" in a VR should expedite information access and learning. Central to this perceptual experience of VR is the poorly understood phenomenon of immersion or presence. Preliminary insight based on the SIMNET experience (Psotka, 1993) provides not only personal testimonials to the motivating and stimulating effects of the social and vehicle-based immersion of synthetic environments, but also preliminary effectiveness data on its potency for learning and training. That is, even though SIMNET provides an impoverished perceptual simulation of a tank in action, the cues from interactive communications among crew members, as well as the auditory and visual cues of the simulated sights, provide gut-wrenching and sweaty believability. What's more, the evidence clearly shows a level of training effectiveness (even without a curriculum) that is superior to many other classroom and simulation-based efforts (Bessemer, 1991). Research is continuing on how to make this training more effective by including surrogate crew members and intelligent semiautomated forces in the environments. The need to involve dismounted infantry, not just tanks and vehicles, is creating a research base for better computational models of agents and coaches (Badler, Phillips, and Webber, 1992).

Virtual reality shows promise in the construction of microworlds for physics and other science instruction. For instance, Loftin and Dede (1993) are creating a Virtual Physics Laboratory from the base facilities of a VR world created for NASA astronaut training. In their virtual laboratory, students can conduct experiments in a virtual world where everyday accidents, structural imperfections, and extrinsic forces, such as friction, can be completely controlled or eliminated. Balls that bounce with complete determinism can be measured accurately at all times and places, and can even leave visible trails of their paths. The effects of gravity can be controlled, and variations of gravity can be experienced visually, and perhaps even kinesthetically.

Although the perceptual aspects of experience are clearly important, it is easy to assume that there are no difficulties to learning from existing visual representations and simulations, like photographs, graphs, and static drawings. It is easy to downplay and overlook difficulties in modern learning environments. Most of us are experts at interpreting visual representations on printed pages (figures, graphs, photographs, icons, drawings, and prints), but it's easy to forget the difficulty we once experienced as we tried to interpret scatter plots and line graphs. We know from many studies that those difficulties never completely go away. For younger learners, they may be even more pronounced. VR can remove these difficulties to a degree and make information more accessible through the evolutionarily-prepared channels of visual and perceptual experience. As to the question of whether the delivered "bang" is worth the bucks, the jury is still out.

We now turn our attention away from these controversies, and toward the analysis of a collection of ITS that have been systematically evaluated and reported in the literature. The purpose of this section is to provide a flavor for evaluations that have been conducted, rather than to review all possible evaluations.