33.5 The Effectiveness Of Learner Control In CBI

Unfortunately, the empirical research on learner control in instructional contexts does not support its unconditional use (Carrier, 1984; Hannafin, 1984; Milheim & Martin, 1991; Steinberg, 1977, 1989). Many authors of texts on CBI design caution against cavalierly offering a variety of options to learners (e.g., Alessi & Trollip, 1985; Jonassen & Hannum, 1987; Steinberg, 1984) because such a strategy does not seem to improve overall learning. O'Shea and Self (1983, Chapter 3) summarize much of the unpublished research on the effectiveness of the early TICCIT system and conclude that it is difficult to support its widespread use. Merrill (1983), too, concludes that college-level students generally do not make good use of learner control options, a position also taken by Carrier (1984). Snow (1980), commenting on the use of learner control in adaptive instruction, argues that far from eliminating the effects of individual differences on learning, providing learner control may actually exacerbate these differences.

Research on learner control in CBI has typically compared learner-controlled and program-controlled treat-ments in a fashion reminiscent of, and analogous to, media comparison studies conducted in the 1960s and 70s (and with analogous methodological problems; see Reeves, 1993, for a thorough critique). The following is an updated summary of research findings comparing learner-controlled computer-based instruction with either partial learner-con-trolled versions or complete computer-controlled versions for the three most common types of dependent variables measured in such studies, namely, learning, time-on-task, and attitudes and affect.

Generally speaking, these studies compare treatments that present a mixture of the specific instructional events actually subject to learner control. Also, contrary to Hannafin's (1984) claim, both adults and children seem to have been well represented in these studies. Many of these studies were also reviewed by Carrier, (1984), Hannafin, (1984), Milheim and Martin (1991), and Steinberg (1977, 1989), but are discussed in this chapter with more stress given to attempting to understand the underlying processes governing students' choices. Additionally, many studies are examined here which have not been included in previous review papers.

33.5.1 Learning

On the whole, results have been mixed, but instructional treatments under the learner's control have been shown most often to be as effective or less effective than treat-ments under more computer control. Also contrary to a suggestion by Hannafin (1984), findings from the literature reviewed here indicate that children do not seem to be less able to handle learner-controlled situations than adults.

A few studies have supported the use of learner control of at least some instructional events (Avner, Moore & Smith, 1980; Campanizzi, 1978; Ellermann & Free, 1990; Kinzie, Sullivan & Berdel, 1988; Mayer, 1976; Newkirk, 1973; Shyu & Brown, 1992). Most of these support Hannafin's (1984) suggestion that learner control promotes a deeper or more long-lasting effect on memory. Newkirk (1973), for example, found a long-term learning benefit for learner control, but not for program control. Mayer (1976) found that more complex outcomes were learned better when learners were able to control the order of presenta-tion, while simple outcomes were learned better under experimenter-controlled conditions. In one of the few long-term studies of the effects of learner control in CBI, Avner et al. (1980) found that students using highly "interactive" learner control showed a greater degree of high-level skills than did the students within a more "passive" type of CBI. There were no differences between these groups on low-level skills, however. In a paired-associate task investigated by Ellermann and Free (1990), students who could select the order of presentation seemed to have a stronger memory trace, implying more engagement of cognitive structures.

In contrast, studies by R. C. Atkinson (1972), Belland, Taylor, Canelos, Dwyer, and Baker (1985), Johansen and Tennyson (1983), Lee and Wong (1989), MacGregor (1988), Morrison, Ross, and Baldwin (1992), Olivier (1971), Pollock and Sullivan (1990), Reinking and Schreiner (1985), Rivers (1972), C. L. Tennyson, Tennyson, and Rothen (1980), Tennyson and Buttrey (1980), Tennyson, Park, and Christensen (1985), Tennyson, Welsh, Christensen, and Hajovy (1985), all found various types or degrees of program control superior to learner control of the same instructional elements for posttest achievement. Many of these authors speak of learners not having or not knowing how to utilize appropriate strategies when they are left to themselves to manage their learning environment.

Interestingly, most studies in which the computer controlled the rate of pacing, that is, the length of time in which screensful of information were presented to the student (Belland et al., 1985; Dalton, 1990; Tennyson, Park & Christensen, 1985; Tennyson, Welsh, Christensen & Hajovy, 1985), found learning under those conditions better than self-paced conditions (in which the learner controls the speed at which material is presented), the usual fixture in most CBI programs. One study by Milheim (1990), how-ever, did find learning better under a learner-controlled pacing condition.

Additionally, in a meta-analysis of 10 years of interactive video instruction, McNeil and Nelson (1991) conclude generally that program-controlled conditions are superior to learner-controlled condition. They suggest however, that partial (i.e., "guided") learner control over review and prac-tice activities might be better for learning than program control over these activities, although they caution that too few studies included such conditions to make the con-clusion unequivocal.

Most studies, however, found no differences overall between learner-controlled and program-controlled treat-ments (Arnone & Grabowski, 1992; Balson, Manning, Ebner & Brooks, 1984/1985; Beard, Lorton, Searle & Atkinson, 1973; Carrier, Davidson, Higson & Williams, 1984; Carrier, Davidson & Williams, 1985; Carrier, Davidson, Williams & Kalweit, 1986; Fredericks, 1976; Goetzfried & Hannafin, 1985; Gray, 1987; Hannafin & Colamaio, 1987; Holmes, Robson & Steward, 1985; Hurlock, Lahey & McCann, 1974; Judd, Bunderson & Bessent, 1970; Judd, O'Neil & Spelt, 1974a; Kinzie & Sullivan, 1989; Klein & Keller, 1990; Lahey, 1978; Lahey & Coady, 1978; Lahey, Crawford & Hurlock, 1976;. Lahey, Hurlock & McCann, 1973; Lee & Lee, 199 1; Lopez & Harper, 1989; McCann, Lahey & Hurlock, 1973; Murphy & Davidson, 1991; Pridemore & Klein, 1991; Relan, 1991; Ross, Morrison & O'Dell, 1988, 1989; Schloss, Sindelar, Cartwright & Smith, 1988; Schloss, Wisniewski & Cartwright, 1988; Strickland & Wilcox, 1978; Wilcox, Richards, Merrill, Christensen & Rosenvall, 1978). The various conclusions drawn from this "no-difference" finding are interesting and tend to reflect a good deal of rationalization. Some of the researchers use this finding to support the use of learner control, saying that programming the computer to handle the myriad complex types of branching that could potentially occur in a lesson is far too difficult. So therefore, since their research indicates it would at least do no harm, it is better to let students handle their own lesson branching. Other researchers use the "no-difference" result to justify program control of instruction, saying that other benefits, such as time savings (discussed next), are realized by not letting learners control their own instructional paths. Still other researchers look more close-ly at their data to discern any interaction effects that may be operating; that is, learner control or program control might be better for some people or under some conditions, but not on the whole. These types of studies are discussed in more detail later in this chapter.

33.5.2 Time-on-Task

Several studies also included as a dependent variable the length of time students took to complete a lesson. A few studies found students in learner-controlled CBI groups taking more time to finish the lesson than program-con-trolled groups. In a study by MacGregor (1988), elementary students worked in pairs, and those in the learner-controlled group were given the opportunity to participate in an on-line instructional game; students in the program-controlled group were not. The author attributed the time-on-task differences to the fact that the game aroused quite a bit of interest, thus generating a lot of talking and other social activity within the pairs, thus naturally consuming more time in the process. Dalton (1990) also found that students in a condition in which the computer controlled the pacing of materials spent more time than those in learner-controlled pacing. He, too, suggests that the amount of socializing observed among the paired members of the learner-controlled condition accounted for the longer time spent. Shyu and Brown (1992) explain that longer times in their study for students under, learner control can be explained by those students needing more time to figure out how to operate the learner control features of the computer program. Another study (Avner et al., 1980) found that while students in learner control conditions spent more time during on-line tasks, they spent less time during related off-line tasks, in this case laboratory activities. Interesting-ly, none of these authors of studies attributed the additional time spent by students in learner-controlled conditions to a greater or deeper degree of cognitive engagement, even though there is theoretical justification for such a conclusion.

A few studies found no differences in time spent (Hurlock et al., 1974; Kinzie & Sullivan, 1089; Lahey et al., 1973). The bulk of studies, however, found that learner-controlled groups spent considerably less time than program-con-trolled groups (Fredericks, 1976; Johansen & Tennyson, 1983; Lahey et al., 1976; Murphy & Davidson, 1991; Rivers, 1972; Ross et al., 1988; Schloss, Sindelar, Cartwright & Smith, 1988; C. L. Tennyson et al., 1980; Tennyson, 1980; Tennyson & Buttrey, 1980).

Researchers investigating "efficiency" of time spent dur-ing a lesson have found mixed results. Dalton (1990) found no differences on achievement-per-time-spent between self-paced and lesson-paced interactive video formats, although Relan (1991) did find such differences in a CBI study favoring the least amount of learner control. Another study (Goetzfried & Hannafin, 1985) did find differences between groups on an efficiency variable they define as the number of concepts a student sees per minute. In their study, however, learner control was the least efficient; that is, it promoted a slower progression through the lesson.

In some of these studies (Goetzfried & Hannafin, 1985; Johansen & Tennyson, 1983; C. L. Tennyson et al., 1980; Tennyson, 1980; Tennyson & Buttrey, 1980), shorter time was also linked to poorer performance. One possible explanation for these findings lies in the confounding of instructional control, time-on-task, and amount of instruc-tional material seen. That is, learners navigating their way through a lesson might spend less time because they opted to skip over large amounts of instructional material. If this omitted material were crucial for overall lesson perfor-mance, these students might naturally be expected to perform more poorly than would students progressing through a program-controlled, but more "complete," lesson package. This notion is also offered by Lepper (1985), who suggests that students under learner control might see differing amounts or kinds of instructional material than students under program-controlled treatments. (In fact, for some situations it is entirely likely that each student under learner control selected his or her way through a completely different instructional treatment!) Therefore, in these stud-ies we do not know whether the culprit for the supposed failure of learner control is the fact that students were grant-ed control, per se, or simply saw suboptimal amounts of instruction as a result of "poor" choices. Indeed, several of these studies do report that students in learner control spent less time because they saw fewer instructional screens (e.g., fewer examples).

Commenting on the problem of learners choosing low amounts of instruction, Higginbotham-Wheat (1988) and Ross and Morrison (1989) draw the conclusion that learner-controlled instruction should allow naive or otherwise unprepared students to select context, sequence, and presen-tation style variables only, and should not allow students to choose instructional events that could alter the amount of content support, unless they have the prerequisite skills or training. It seems safe to say that the confounding of learner control, time-on-task, and amount of instructional material is a matter that can prevent clear conclusions about the relative merits of offering control to learners. This issue .surfaces again later in this chapter.

As with the findings of no difference in learning between learner- and program-controlled groups mentioned earlier, the authors of the studies who found shorter times for learner control also are quite creative in the implications they draw from that finding. Some authors claim that if learning is equivalent, but time spent is shorter, learner control is desired because of the time "savings" or "economies." Others say that the shorter times from the learner control groups mean less time-on-task and time for cognitive processing, inherently undesirable outcomes, and thus should be avoided.

33.5.3 Attitudes and Affect

Most of the studies that measured the students' attitudes toward computer-assisted instruction or toward learner control over instruction found either no differences or a favorable attitude from students who experienced a learner-controlled treatment compared with program-controlled groups. The author of the one study that did find relatively negative attitudes in the learner control group (Gray, 1987) explained the findings as due to resentment and frustration at the complexity of the instructional decisions the students under the learner-controlled condition were required to make.

A few studies (Arnone & Grabowski, 1992; Beard et al., 1973; Judd et al., 1970; Kinzie et al., 1988; Lahey, 1978; Pridemore & Klein, 1991; Shyu & Brown, 1992) found no differences between learner-control and program-control groups in student attitude toward CBI. Additionally, one review (Judd, 1972) also concluded that, generally, learner control does not contribute to improved attitudes. However, it is quite possible that the early computer studies included in Judd's paper do not represent the highly interactive approaches used for instruction on modem microcomputers.

Six studies did find positive attitude effects for students in learner-control groups (Hintze, Mohr & Wenzel, 1988; Hurlock et al., 1974; Judd et al., 1974a; Milheim, 1989; Morrison et al., 1992; Newkirk, 1973). For example, the study by Milheim (1989) exploring interactive videodisc instruction found better attitudes toward the instructional activity for students under learner-control pacing compared with students who experienced program control of lesson pacing. In most of the studies examining attitudes, students were exposed only to one type of program (i.e., they only saw a learner-controlled version or a program-controlled version). One of the most interesting of all the studies examining attitudes is presented by Hintze et al. (1988), who compared attitudes in dental students in Denmark, each of whom actually had a chance to experience several versions: completely learner-controlled, partially learner-controlled, and computer-controlled instructional situations. They found the overwhelming majority of students preferred at least some learner control. Interestingly, males by far preferred complete learner control, while female prefer-ences were split between partial and total learner control. They suggest possible explanations for this finding, includ-ing that it might be due to the fact that males typically have spent more time with computers, and thus are more likely to want to explore with them when given the chance. Dalton (1990) also found some interesting interactions between gender and learner- or program-controlled treat-ments on attitudes. Specifically, he found that females under lesson-controlled pacing ended up with better attitudes toward instruction and toward the content of the lesson than females under learner-controlled pacing. Males, however, under program-controlled pacing had significantly worse attitudes toward content than males under learner-paced lessons. However, convincing explanations for these gender effects and interactions were not provided by the authors of these studies and remain to he drawn, particularly as they relate to the underlying social, motivational, or cog-nitive processes involved. For example, it is possible that males have a greater comfort level (less anxiety) when they sit at the computers, and thus feel more confident to "take charge" of them.

Some researchers investigated the effects of learner-controlled instruction on one particular type of attitude measure called continuing motivation (Maehr, 1976; Seymour, Sullivan, Story & Mosley, 1987), which indicates how likely is a student's ongoing willingness to return to a learning activity at a later time without external pressure, essentially a variable measuring the student's desire to learn voluntarily. Kinzie and Sullivan (1989) found positive effects on the students' desires to pursue science activities following computer-assisted instruction, generally, and following learner-controlled CAI, specifically. However, this effect was not replicated by Lopez; and Harper (1989), who found no advantage for learner control over program control on continuing motivation in science. Lopez; and Harper (1989) do not offer any specific explanation to reconcile this discrepancy, but they imply that there might be characteristics of the population studied (high-risk Hispanic students), which had a unique bearing on the find-ings in their study.

Lastly, a few early studies investigated the effects of learner-controlled computer-based instruction on student state (i.e., temporary) anxiety, with mixed results. Judd (1972) recaps one study that shows a reduction in anxiety as a result of learner-controlled CBI. J. B. Hansen (1974) also found a lowering of initial state anxiety as a result of learner control over computer-delivered feedback. However, neither Judd, O'Neil, and Spelt (1974b), nor Judd, Daubek, and O'Neil (1975), were able to lower state anxiety as a result of their particular types of learner control (control over access to mnemonic devices in the first case, and control over access to pictures in the second case).

33.5.4 Summary of the Effectiveness of Learner Control of CBI

After reviewing all of these findings for the various types of dependent variables, we are presented with an apparent dilemma: Given the rationale for learner control provided earlier, there are good reasons to believe that learner control is a desirable instructional approach. However, the bulk of studies conducted to assess that notion found that students left on their own do not uniformly make good use of such strategies. Duchastel (1986b) sums up the frustrating ambi-guity of learner control research:

... the research leads one to be cautious about the general learner control hypothesis, namely, that the student is the best judge of the instructional strategy to be adopted. Some results in instructional research indicate that not all students are capable of making appropriate educational decisions. Other results, however, indicate the tremendous benefits of learner control in particular situations. The sophistication of the learner and the type of objectives pursued, as wen as the particular context of the system, will probably impact on the nature and effectiveness of learner control in given situations (p. 391).

But there are problems with the research itself, which also limits the conclusions we might draw from this body of literature. Reeves (1993) presents a very thorough cri-tique of the bulk of learner control research, suggesting that the lack of unambiguous findings is in fact due to bad research. He lists a variety of shortcomings with the litera-ture spanning four crucial areas. First, he presents some problems in many studies with the definition of learner control used. That is, many authors fail to provide adequate operational definitions of their learner control treatments, and so end up with, at best, muddy and ambiguous experi-mental designs. Secondly, he discusses the lack of adequate theoretical foundations undergirding the experiments. Studies often proceed, he says, without even a tenuous con-nection with the literature of basic learning or instructional .theories. (Whether this type of uninformed research activity is necessary for the "theory-building" phase required of any school of systematic scientific investigation, or instead bluntly demonstrates intellectual expedience or laziness on the part of researchers, is a matter of separate debate.) The last two problem areas he finds in the learner-control litera-ture are methodological and analytical, with researchers too seldom using adequate designs (often including poor instrumentation, control of procedures and sampling; see 39.2. 1) and procedures for analyzing the data (e.g., improp-erly employing quantitative paradigms when qualitative approaches would make more sense; see 39.4.1.2).

As an example, a typical problem in learner-control research is the confounding of learner-control or program-control treatments with the amount of instruction students see during the lesson, a problem mentioned earlier in this chapter. That is, as Lepper (1985) points out, the often-repeated failure to demonstrate die effectiveness of learner control might simply be a function of the fact that less instructional material is selected by those students; hence they received an "incomplete" lesson compared with their program-controlled counterparts. In other words, learner-control ' ineffectiveness would be an artifact of the particular set of instructional events they experienced (or, more likely, did not experience).

Indeed, Ross and Rakow (1981), C. L. Tennyson et al. (1980), Tennyson (1980), and Tennyson and Buttrey (1980), all showed that students in the learner-controlled treatments saw many fewer instructional examples than did students under program control. However, in these studies, lower amounts of instructional material were inextricably confounded with the learner-control. treatments. Carrier and Williams (1988) experimentally controlled the amount of material seen, and found a positive effect for amount of material separate from learner-control or program-control effects. In a study by Morrison et al. (1992), amount of instructional material was controlled for by having two program-controlled versions: one with "minimum" instructional support, one with "maximum." They found that the students under learner control actually performed poorer than those with the "minimum' program-control treatment. The type of studies that attempt to control for amount of instruction separately from learner control have been fairly rare, but they do allow for a clearer examination of the effects of learner choices independent from amount of instruction.

These criticisms also prevent researchers from conducting statistical meta-analyses on the literature as a way to provide a little more concrete basis for making statements about the effectiveness of learner control. Because the studies about learner control are so incredibly varied (each study seems to offer unique operationalizations, instrumentations, designs, and analyses), it would be practically impossible to arrange the studies into sets of reasonably equivalent dependent and independent variables, and to conduct a meta-analysis so as to avoid the concerns Slavin (1986) has voiced about improper use of the meta-analysis technique.

Similarly, a "best-evidence" approach (Slavin, 1986) to synthesizing the learner-control literature would also be difficult to defend, because the criteria for inclusion of studies in such a synthesis (e.g., relevance, minimal bias, external validity) are frequently not met, as Reeves (1993) has forcefully presented. If a best-evidence synthesis were attempted, it is likely that after gleaning from the literature the studies worthy of examination, there might not b~ that many "clean" articles left that could provide any useful information for inclusion in the synthesis.

If, as seems amply demonstrated by Reeves (1993), there are problems that prevent us from drawing confident conclusions from the domain of learner-control research in CBI, the next logical question would be, "Is there anything we can say from looking at these studies" Depending on the research perspective taken, reviewers might reject the body of research to date (because of the flaws); or they might conduct a type of salvage operation, examining the studies post mortem. for clues and commonalties across the papers that might help both explain the diverse findings previously cited, and to provide new researchable questions. Kinzie (1990), for example, provides a very helpful mapping of some research findings in the learner-control literature, with various theoretical perspectives in motivation, individual differences, social psychology, and self-regulated learning. However, as with all post hoc types of examinations of literature (including this chapter), it is important to stress that conclusions drawn should be considered in large part to be conjectures, inferred from the studies, and which remain to be a priori tested in their own right.