32.5.1.1. Complexity. Feedback complexity refers to how much and what information should be included in the feedback messages. There is an abundance of literature concerning feedback complexity. Dempsey, Driscoll, and Swindell (1993) have organized the major variables of interest in most corrective feedback studies as follows:

If feedback is to serve a corrective function, even the simplest feedback should verify whether or not the student's answer is right or wrong. This verification is usually combined with an elaboration component in order to provide more information to the learner. Studies that have examined the type and amount of information in feedback have not yielded very consistent results (Kulhavy, 1977; Schimmel, 1988).

What types of elaborative information have been used along with the verification component in the feedback message? In a review of the feedback literature, Kulhavy and Stock (1989) suggest that there are basically three possible elaboration types to employ during feedback. They categorize them as: (1) task-specific, which is drawn from the initial task demand or initial question (e.g., restatement of the correct answer); (2) instruction-based, which contains information derived from the specific lesson material, but not directly from the actual question completed before the feedback (e.g., explanation of why an answer is correct, based on the original instruction, or a display of the original instructional text that contains the correct answer); and (3) extra-instructional, which is the addition of information from outside the immediate lesson environment (e.g., new information to clarify meaning). The majority of elaboration studies fall within the task specific and instruction-based types.

First, consider task-specific types of feedback, where the feedback is a restatement of the correct answer. Usually studies that contain this type of feedback have examined changes in the amount of information, sometimes referred to as load. A study by Phye (1979) examined three types of feedback for multiple-choice questions. One contained the question stem and only the correct alternative; another contained the stem and designated correct answer with incorrect alternatives from the question; and a third contained the stem with designated correct answer with the two incorrect alternatives from the question plus two previously unseen incorrect alternatives. No differential effect was produced by type of feedback on the posttest. However, in a second experiment in the study, immediate feedback in the form of only the correct answer plus an answer sheet from the practice was superior to other forms of feedback. Thus, the type of feedback thought to provide the least information produced the greatest improvement on the posttest. Phye suggests a threshold hypothesis to account for this unexpected finding, positing that when more than sufficient information needed to correct or confirm an answer is provided to the student, it does not have a facilitative effect on his ability to use the feedback.

Some studies that have added increases of task information to feedback have actually produced lower scores on a posttest. Phye and his associates (Phye, Gugliamella & Sola, 1976) used feedback very similar to that used in the Phye (1979) study, adding either the correct answer only, the initial item plus all original distracters, or the correct alternative and three extra-list distracters. Feedback in the form of correct answer only was superior to the other types that contained more information. This would imply that the feedback with more load contained considerable distracting information in the form of incorrect alternatives.

Another similar finding was provided by Sassenrath and Yonge (1969) in providing two types of feedback cues: with or without the stem of the question, and with or without correct plus wrong - alternative answers. Students who received information feedback without the stem of the question performed better than those who received information feedback with the question stem. This refutes the results of a previous study they completed (Sassenrath & Yonge, 1968), in which students receiving the stem of the question and the alternatives performed better on a retention test than those only receiving the alternatives. The researchers explain this discrepancy by the fact that the earlier (1968) study gave feedback after the students had responded to the entire list of questions, so that the question stem conveyed valuable information in addition to the alternatives, But in the second study (1969), feedback was presented after each item response, and it is suggested that the stem was distracting when used in feedback given within such a short time lapse after a response.

Wending (1973) compared the effects of (1) partial feedback that contained knowledge of results, to (2) total feedback that contained knowledge of correct answer and required a re-response, or (3) no feedback at all. The partial feedbEick treatment exceeded the other two treatments on immediate achievement scores, and, surprisingly, the total feedback treatment was least effective in terms of immediate achievement.

Another study (Hanna, 1976) comparing partial feedback, total feedback, and no feedback found that partial feedback produced highest scores for high-ability students, and total feedback produced the highest scores for lower-ability students. There were no differential effects between partial and total feedback for middle-ability students, but both of these types of feedback were superior to no feedback.

Three studies do show positive results for task-specific item elaborations. Roper (1977) provided students with either no feedback, yes-no verification, or an opportunity to restudy the correct answer Scores on the posttest increased as more information was added to the feedback. There was also evidence that the correction of errors and not just reinforcement of responses was the major effect of feedback. Also, Winston and Kulhavy (cited in Kulhavy & Stock, 1989) found that using feedback consisting of a multiplechoice item stem plus the correct response and all of the original distracter alternatives was more effective at correcting errors than when using feedback containing the stem plus only the correct alternative. And finally, an early study (Travers, van Wagenen, Haygood & McCormick, 1964) gave an interesting variation of task-specific feedback for corrects and wrongs. One group received verification for both corrects and wrongs; a second group received verification only for wrongs and nothing for corrects; a third group received verification only for corrects and verification plus the correct answer for wrongs; and a fourth group received nothing for corrects and verification plus the correct answer for wrongs. A relationship between information content of the feedback condition and extent of learning was found to exist. Highest criterion test performance occurred under the last two feedback conditions-the ones that were the most information laden. The second feedback condition of merely saying "That's wrong" was significantly inferior to all other conditions studied.

An even more inconsistent pattern of results is found in studies that have used instruction-based elaborations, in which information in the feedback is taken from the instruction itself. The information used in this type of feedback has been quite diverse, including explanations of the correct answer (Gilman, 1969), supplying solution rules (Birenbaum & Tatsuoka, 1987; Lee, 1985; J. Merrill, 1987) and re-presenting original instruction ( Peeck, 1979).

Gilman (1969) employed "additive" feedback, comparing (1) no feedback, to (2) feedback of "correct" or "wrong," (3) feedback of . correct response choice, (4) feedback appropriate to the student's response, or (5) a combination of (2), (3), and (4). The means of the groups that had guidance toward the correct answer [groups (3), (4), and (5)] Performed better than the groups who had to search for the correct answer. Gilman points out that providing learners with a statement of which response was correct or with a statement of why the correct response is correct may be of n1o" "IM than YnerelY telling the learner "correct, or wrong." In terms of error correction, knowledge-of-results feedback resulted in the least number of corrected errors. In terms of retention rates, Gilman suggests that extensive information in feedback messages show advantages in retention rates.

Merrill employed both corrective feedback and attribute isolation feedback in his 1987 study of feedback to aid concept acquisition. Corrective feedback informed the learners of the correctness or incorrectness of their answers and also provided the full text of the correct answer when a student's answer was wrong. The full text consisted of a single word, phrase, or short paragraph. Attribution isolation feedback also informed the learners of the correctness of their responses, but then included the attributes of the concepts being studied. Attribution isolation is used to help focus attention on the variable attributes of a concept (M. Merrill & R. Tennyson, 1977). No main effects for feedback form were found, possibly due to the attribute isolation feedback being presented after two incorrect responses and, consequently, not being encountered enough times in the lesson to make a difference.

Another study (Lee, 1985) that provided solution rules in its feedback used either (1) "right/wrong" feedback only, (2) "right/wrong" plus the correct answer after an error, or (3) "right/wrong" plus the rule restated and the correct answer after an error. No significant main effects were found in the feedback treatments.

One unique approach using feedback solution rules was devised by Tatsuoka and her colleagues (cited in Kulhavy & Stock, 1989). The seriousness of instructional errors were analyzed from a pretest in order to assess the effect of additive feedback elaborations on a later criterion measure. Students received feedback as either (1) "OK/No" verification, (2) the correct answer to the problem, or (3) a statement of correct and incorrect rules for solving the problem. They found that for nonserious errors, more feedback elaborations result in a greater probability of these errors being corrected. But for serious errors, correction was relatively unaffected by the amount of elaboration. This finding suggests that more complex errors or misunderstandings are not as likely to be corrected by typical feedback treatme its.

Schloss and his colleagues (P. J. Schloss, Sindelar, Cartwright & C. N. Schloss, 1987) presented either instructions to try again or a re-presentation of the instruction after student errors in computer-assisted instructional modules to test if error correction procedures would interact with question type, such that higher-cognitive questions with feedback loops and factual questions with re-Iiresentation of questions would yield maximum results. They concluded that when factual questions are used in CAI modules, allowing a student to attempt a second answer after an error results in more learning than re-presenting the part of the instruction in which the answer appears.

Sassenrath and Garverick (1965) compared more traditional classroom types of feedback: looking up wrong answers in the textbook to having answers discussed by the instructor or checking over answers from correct ones written on the board. These three feedback groups did perform significantly better on a retention test than a no-feedback control group. The discussion group also performed better than the groups that looked up answers in the textbook.

Students in a different study (Peeck, 1979) were either given feedback sheets identical to immediate test sheets, with the correct alternatives circled or were given both the original text and the feedback sheets with correct alternatives circled. Also, to test if the effectiveness of different forms of feedback was influenced by the kind of test question presented, both fact and inference multiple-choice questions were used. There was little difference in scores between the two feedback conditions. More inference questions were answered correctly when subjects could refer to the original text during the feedback. But for fact questions, subjects were more successful on a delayed test when the text was absent during the feedback.

Similarly, two types of questions (factual and application) and two types of feedback (correct-answer feedback, self-correction feedback, and no feedback control) were employed in a study by Andre and Thieman (1988). Both types of feedback facilitated performance on the same concept questions but did not facilitate the application to new examples. This suggests that such feedback may be helpful in tasks where the students memorize an answer but be ineffective for tasks that require application to new cases.

Even large-scale additions to the feedback have failed to influence posttest performance, as was the case for Kulhavy and his colleagues (Kulhavy et al., 1985). Four types of feedback were developed additively. Four components could be used in the feedback: (1) test item stem and the correct alternative; (2) incorrect response alternatives; (3) four sentences, each explaining why one of the error choices was incorrect; and (4) the relevant section of the passage in which the correct answer was identified. One group received only component number (1); a second group received components (1) and (2); a third group components (1), (2), and (3); and a fourth group all four components. The principle was that increases in the feedback complexity are closely tied to corresponding increases in the amount of information available to the learner. Results showed that more complex versions of feedback had a small effect on error correction, with the least complex feedback correcting a significantly greater portion of errors than the more complex third feedback group.

In a CAI drill-and-practice program using a conceptlearning task, it was indicated that immediate extended feedback following both correct and incorrect responses is superior to minimal feedback (Waldrop, Justen & Adams, 1986). In the first of three treatment conditions, subjects received only minimal feedback of "correct" or "incorrect." In a second treatment condition, subjects received minimal feedback ("that's correct") if a response was correct, but received minimal feedback ("that's incorrect' ') for three hials if a response was wrong. After the third trial, if a response was still incorrect, students were provided extended feedback relating the example given to the definition of the type of consequence involved in that example. The third treatment condition provided a detailed explanation of the correct answer following both correct and incorrect responses. The results of this study agrees with a suggestion made by Gilman (1969), that providing the student with a statement of which response was correct after errors and providing reasons for correctness of a correct response are essential.

Noonan (1984) examined the presence or position of knowledge of results (KR), knowledge of correct response (KCR), and elaborated and try-again feedback. In this study, knowledge of results with an explanation and a second attempt was no less effective than giving KCR and moving on or giving KCR and another second attempt. In support of error analysis, Noonan suggests that explanations should depend more on the type of error made by the learner and not merely on the correct answer.

Varying types and amounts of information in feedback given after specific combinations of answer correctness and response certitude in a CAI lesson was used by Chanond (1988). If a subject's answer was correct, and he or she was confident of the answer, the subject received knowledge of result feedback. If a subject's answer was correct, but he was not confident of his answer, he received knowledge of result and a statement of why the response was correct. If a subject's answer was incorrect, but she was confident of her answer, she received knowledge of result, a statement of why the response was incorrect, knowledge of correct response, and a statement of why the correct answer was correct. If a subject's answer was incorrect, and he was not confident of his answer, he received knowledge of result, knowledge of correct response, and a statement of why the correct answer was correct.

Subjects were given both an immediate and delayed posttest at the end of the lesson. Results indicated that for immediate retention of verbal information in terms of overall correct responses, the feedback had a significant effect No significant effect was found for delayed retention, however. Further analyses indicated that, regardless of the level of confidence for the response, feedback following incorrect responses had a significant effect on both immediate and delayed retention.

The use of extra-instructional feedback types has been studied very little (Kulhavy & Stock, 1989). However, adaptive feedback that additively used all three typestask-specific, instruction-based, and extra-instructional feedback-was implemented by Mory (1991) and involved two levels of learning tasks: verbal information and concepts. Varying combinations of task-specific, instruction-based, and extra-instructional feedback were prescribed according to a combined assessment of answer correctness and response certitude level for an adaptive feedback group. When compared to nonadaptive feedback that utilized taskspecific and instruction-based elaborations only, there were no significant differences in posttest performance for either verbal information or concept tasks.

To summarize the feedback elaboration literature, only half the studies utilizing task-specific feedback produced any significant improvements in learning. An even greater inconsistency is found in studies using information-based feedback, perhaps partially due to the diverse types of information manipulations tried. Such variance has made it difficult to prescribe any set rule for the use of either type of elaborations (Kulhavy & Stock, 1989). Extra-instructional feedback types have not been researched enough to draw conclusions as to their effectiveness on learning.

Recall from the early reports of feedback research that the idea of feedback as reinforcement-a Skinnerian view would suggest that feedback should follow a response as closely in time as possible in order to be most effective (see 2.2.1.3.2). Skinner himself is quoted as saying, ". . . the lapse of only a few seconds between response and reinforcement destroys most of the effect" (cited in Kulhavy & Wager, 1993, p. 13). But when researchers began comparing the effects of immediate versus delayed feedback, discrepancies from such an operant approach were soon discovered. Kulhavy (1977) reports that studies showed repeatedly that delaying the presentation of feedback for a day or more results in significant increases in student retention on posttest scores (Sassenrath & Yonge, 1968, 1969; Sturges, 1969, 1972). This phenomenon was termed the Delay-Retention Effect (DRE) (Brackbill, Bravos & Starr, 1962; Brackbill & Kappy, 1962) and was found to occur predominantly in studies concerned with multiple-choice testing. The explanation for the DRE is thought to lie in the proactive interference from initial error responses upon the acquisition of correct answers given via immediate feedback. That is, when a learner is presented immediate feedback showing the correct response after an error, his or her error response interferes with the correction of the response due to the immediacy of the feedback. Thus delayed feedback eliminates this type of interference, and the learner is better able to remember the correct response. Several studies support this hypothesis: interference-perseVeration hypothesis, explaining the DRE through the assumption that initial errors tend to be forgotten over time (Bardwell, 1981; Kuthavy & Anderson, 1972; Kulik & Kulik, 1988; Sassenrath, 1975; Surber & Anderson, 1975). But others have found that either the delay did not make a difference (Peeck et al., 1985; Phye et al., 1976), that initial responses were not forgotten (Peeck & Tillema, 1979), or that the DRE was not present when subjects were required to re-respond (Phye & Andre, 1989).

In a 1988 meta-analysis conducted by Kulik and Kulik, the issue of immediate versus delayed feedback was examined more thoroughly. In analyzing the available research on the timing of feedback, they found that studies using actual classroom quizzes and materials usually found that immediate feedback was more effective than delayed feedback. Apparently the studies that supported the effects of delayed feedback over immediate feedback for improving retention of material were conducted using contrived, experimental learning situations, such as list learning. These findings challenge both the use of delayed feedback in more practical learning environments and the explanations afforded by the interference-perseveration hypothesis in "real-world" learning situations. Dempsey, Driscoll, and Swindell (1993) suggest that delaying feedback in many instructional contexts "is tantamount to withholding information from the learner that the learner can use" (p. 24). And a pragmatic suggestion postulated by Tosti (1978) and Keller (1983) is to present feedback containing pertinent information from the learner's prior performance right before the next learning trial, when the learner would be able to use the information to improve his or her subsequent learning, As Dempsey and his associates (Dempsey, Driscoll & Swindell, 1993) point out, this amounts to providing feedback at what is commonly referred to as "the teachable moment" (p. 24). An interesting variation involving a delay of feedback was designed by Richards (1989) using a declarative knowledge task involving labels and facts. In this case, feedback was more effective when delayed temporarily and when the learner was required to respond covertly a second time to the question-that is, a covert second try, prior to feedback.

In a 1989 study conducted to examine the timing of feedback with respect to the acquisition of motor skills, shorter feedback times improved acquisition and performance while feedback was present, but delayed feedback resulted in improved subsequent performance once feedback had been withdrawn (Schmidt, Young, Swinnen & Shapiro, 1989). They explain these findings as what is termed the guidance hypothesis, which suggests that during the initial stages of skill acquisition, immediate feedback guides the learner and results in superior initial performance. But this guidance can lead to dependence on the feedback and obscure the need to learn the secondary skills (such as detection and self-correction) necessary to perform the task without feedback (Schmidt et al., 1989).

The guidance hypothesis is supported by a previous study that examined the effects of immediate versus delayed feedback within the context of an adventure game on subsequent performance (Lewis & Anderson, 1985). Subjects that received immediate feedback were more likely to select appropriate operators, but those that received delayed feedback were better able to detect errors. But a differing trend was found by Anderson, Conrad, and Corbett (1989) when assessing the effects of immediate and delayed feedback within the context of the GRAPES LISP Tutor. Subjects receiving immediate feedback moved through the material more quickly than did those subjects receiving delayed feedback, but there was no significant difference in test performance. A more recent study by Schooler and Anderson (1990) found that when students were acquiring LISP skills, subjects receiving immediate feedback went through the training material in 40% less time than those receiving delayed feedback, yet with no detrimental effects on learning. In a second experiment during the same study, subjects were using an improved LISP editor and less supportive testing conditions. During this trial, subjects in the immediate feedback group completed the problems 18% faster than those in the delayed feedback group, but they were slower on the. test problems and made twice as many errors. A final experiment, a partial replication of the first two experiments, indicated that delayed feedback was an advantage in terms of errors, time on task, and the percentage of errors that subjects self-corrected. They suggest that immediate feedback competes for working memory resources, forcing out necessary information for operator compilation-a finding that would support the interference-perseveration hypothesis mentioned above. In contrast, delay feedback in the study fostered the development of secondary skills such as error detection and self-correction (Schooler & Anderson, 1990).

In terms of what to recommend in terms of immediate versus delayed feedback, as several researchers concur (Dempsey, Driscoll & Swindell, 1993; Kulhavy, 1977; Kulik & Kulik, 1988), in most learning situations delayed feedback appears to function to hinder the acquisition of needed information. Only under very special experimental situations has the use of delayed feedback helped learning. As Kulik and Kulik (1988) point out,

The experimental paradigms that show superiority of delayed feedback are very similar to paradigms used for testing effects of massed versus distributed practice. When experiments deviate from this paradigm, they show results similar to those in applied studies. In such experiments, immediate feedback produces a better effect than delayed feedback does (p. 94).

One only has to look at the myriad of definitions that past researchers have used in each of the areas of immediate and delayed feedback to understand why this area of study has been muddied throughout the research. Dempsey and Wager (1988) have summarized the types of immediate and delayed feedback as shown in Figure 32-3.

Some researchers suggest that as newer technologies offer more instructional delivery options and a wider variety of modalities through which to deliver feedback, these issues will become even more complex (Dempsey, Driscoll & Swindell, 1993). Perhaps as delivery options increase, researchers will be better able to determine when delayed feedback might aid learners.

In the early 1930s, Thorndike demonstrated that errors made in rote learning tasks tend to persist. By the year 1958, Skinner argued that errors made within programmed instruction will tend to persist as well. Elley (1966) tested the hypothesis that errors play different roles in rote and meaningful learning tasks. Results supported the hypothesis, showing that fewer errors were associated with better retentiontion in rote tasks but not in meaningful types of learning. Both experiments supported the hypothesis that errors are undesirable in rote leaming and tend to be repeated even with immediate feedback. However, when learners were given meaningful problems, incidence of errors was unrelated to ultimate performance.

Figure 32-3. Immediate and delayed feedback with CBI: definitions and categories. (From Dempsey & Wager, 1988.)

The current view considers an error to be a valuable opportunity to clarify misunderstanding in the leamer. Thus, errors play an important role in feedback studies today. The belief that feedback's main function lies in correcting errors makes error analyses more critical for gaining insight into the corrective process.

Kulhavy and Parsons (1972) examined errors that are never corrected, or that "perseverate" to a postitest. They suggest that error perseveration is a function of at least three factors: (1) the rated meaningfulness of the items used, (2) the amount of incorrect material available during learning, and (3) the response mode required of the learrier. In their study, students were forced to respond incorrectly to see if these errors would be repeated on a postitest. But their analyses revealed that forcing a student to make an error does not automatically result in the transference of that error to the posttest.

Patterns of pretest-posttest responses were introduced in a limited way by Phye and his colleagues (Phye et a)., 1976). This work was later extended to include three error types (Peeck & Tillema, 1979; Phye, 1979). An error analysis model was developed independently by Peeck and Tillema (1979) and Phye (1979), and this model has been used by several researchers (Peeck, 1979; Phye & Andre, 1989; Phye & Bender, 1989). Their research has served to help further understand how feedback is being used by learners in most experimental settings.

Whenever informative feedback is used in a pretestfeedback-posttest design, five possible outcomes for pretest-posttest response sequences exist. First, when feedback has a confirmatory function, the feedback serves to confirm a correct answer at pretest (a combination sequence of Correct -4 Correct). Secondly, when feedback has a corrective function, it serves to correct an error made on the pretest (a sequence combination of Wrong -~ Correct). And finally, feedback can have no ftinction, as in cases when errors result on the posttest (Phye & Bender, 1989).

The three error types where feedback is considered nonfunctional are described as follows. One type is a same error and is perseverative in nature. A same error occurs when an initial incorrect response reoccurs on the posttest, regardless of any correct answer feedback that was provided. A second type of error is a different error, in which an item is missed on both the pretest and posttest, but was not the same error across trials. That is, the posttest error was a different error than the pretest error. Perhaps insufficient information was encoded during feedback, so that on the posttest the learner remembers that his or her initial response was wrong, but he or she does not remember information well enough to respond correctly. The final type of error is a new error, in which an item was initially correct on the pretest or practice but for some reason was changed to a wrong answer, or new error, on the posttest. Perhaps in this instance, the initial response was a lucky guess, feedback was basically ignored, and a new error resulted on the test.

Thus, the five possible combinations of pretest -4 posttest responses are: (1) Correct -4 Correct, (2) Wrong ---) Correct, (3) Wrong -4 Same wrong, (4) Wrong -4 Different wrong, and (5) Correct ---) New wrong (see Fig. 32-4).

When put into a response pattern profile in terms of percentage of occurrence, a more exhaustive account of test Performance is facilitated (Peeck et al., 1985). Response Pattern profiles have been used for multiple-choice formats (Peeck, 1979; Phye, 1979). Some researchers (Peeck et al., 1985) argue that in order to interpret the cognitive processes involved in such sequences, it is important to determine to what extent learners remember their initial responses after the pretest. Peeck et al. (1985) included "guess questions" that could not be answered from the text and "factual questions" that could be answered from the text. The most important finding was that learners remembered their initial responses in the wrong-changed-to-correct category. This indicates that retention of initial responses did not prevent subjects from leaming the correct answer from feedback, casting serious doubt, incidentally, on the assumption that subjects tend to forget their responses on the initial task after a delay and that error tendencies interfere with learning the correct answers from feedback-an assumption that was a major component of the interference-perseveration interpretation of the delayed-retention effect studies (Kulhavy & R. C. Anderson, 1972). Data also indicated that when subjects changed their initial response after feedback (correct to a new wrong, wrong to correct, and wrong to a different wrong), the highest identification scores were obtained in the category of corrected errors (wrong to correct).

The construct validity of error analysis was addressed by Phye and Bender (1989) and demonstrated when Peeck et al. (1985) examined pooled data from four previous experiments (cited in Peeck et al., 1985). Proportional frequencies for the three error types when averaged across the four studies were .10 for same errors, .06 for different errors, and .05 for new errors. These averages were quite similar when compared to results of Phye and Bender (1989) in which same errors equaled .08, different errors equaled .05, and new errors equaled .04. These data contribute to the construct validity of the error analysis model and suggest its value when combined with correct response and conditional probability data to assess feedback effectiveness.

Further research from an information-processing perspective should address feedback effectiveness and efficiency by considering not only correct responses but also an analysis of processing errors (Phye & Bender, 1989). Error data, when used with correct response data and conditional probability data, "provides a multivariate account of feedback utilization by the learner in a learning situation involving practice" (p. 109). Error data, when used with correct response data and conditional probability data, "provides a multivariate account of feedback utilization by the learner in a learning situation involving practice" (p. 109).

Figure 32-4. Five response pattern combinations, based on Phye and Bender response pattern analysis (1989).

Another way of analyzing errors is to classify them in some way that is related to the specific learning outcome involved. In rule-using tasks, an example would be the classification of errors as "serious" or "nonserious," as was done in an analysis developed by Tatsuoka (see Birenbaum & Tatsuoka, 1987). The measure of seriousness of error types indicated to what extent a wrong rule deviates from the right rule. Using an "error ve&oe' system to analyze signed-number problems, error codes were developed based on the absolute number operation and the sign operation involved in solving problems. Students' response patterns to test items were then classified into three categories: serious errors, nonserious errors, and correct answers.

In concept learning, errors are categorized according to three kinds of concept classification errors: overgeneralization, undergeneralization, and misconception (cited in R. D. Tennyson & Cocchiarella, 1986). When students are learning to classify a member of a concept class, they must make discriminations between examples and nonexamples of the concept. Certain nonexamples may be quite difficult to discriminate from a given concept example (termed a close-in nonexample), and others may be easy to discriminate from an example (termed a far-out nonexample) (Dempsey, 1988). When a learner is consistently making a particular overgeneralization error of accepting nonexamples, it is likely that he is having a problem with fine discrimination of the concept. Fine discrimination errors occur when close-in nonexamples are classified by the learner as an example of a concept. But if the student is regularly classifying a far-out nonexample as a true example, he may be undergeneralizing by rejecting the examples, resulting in an error of gross discrimination. In general, fine discrimination errors result from classification problems on close-in nonexamples, whereas gross discrimination errors result from a student's having classification problems on far-out nonexamples. Since close-in nonexamples are more difficult to discriminate from examples than are far-out nonexamples, more close-in errors (or fine discrimination errors) should be expected to occur. This indeed was the case in a study by Dempsey (1988). In the same study, it was found that learners who made fewer fine discrimination errors during instruction scored significantly higher on a retention test. In fact, 4 out of 10 errors made during the instruction were those that were predetermined as fine discrimination errors, These findings encourage the analysis of close-in and far-out nonexamples associated with fine and gross discrimination errors when employing concepts-learning tasks.

Finally, Meyer (1986) identifies four errors reflected in a review of research on teachers' correction of students that include (1) lack-of-information errors, (2) motor errors, (3) confusions, and (4) rule application errors. Lack-of-information errors result when student's mistakes are caused by missing knowledge. Motor errors result when a student knows the information but cannot express it. Confusions occur when students fail to discriminate correctly between concepts or ideas. And rule application errors result when students apply rules incorrectly in problem-solving situations. Meyer asserts that feedback should be designed to fit each type of misunderstanding.

Since the correction of errors appears to be where feedback has its most promising effects, researchers should continue to examine ways in which to manipulate feedback to maximize this outcome. As Noonan (1984) points out, more sophisticated procedures that involve analysis of common errors or error patterns might be more useful than traditional correct-answer feedback. Adaptive-feedback information can easily be facilitated within a computerbased instruction environment, where the computer can record and analyze the types of errors being made and give appropriate feedback based on error types.

A detailed overview of suggested feedback for various learning outcomes has been offered by Smith and Ragan (1993). These researchers discuss their views of what information to include for each type of learning outcome according to Gagn6's taxonomy (see 18.3). Instructional design theorists have proposed that different types of learning tasks require different strategies and instructional methods (Gagn6, 1985; Merrill, 1983; Reigeluth & Stein, 1983) (see 18.3, 18.4). Very few researchers have attempted to investigate the differences in feedback needs for differing types of learning. Schimmel (1983) found differences in informative feedback given for declarative knowledge versus procedural knowledge. The studies that have been conducted are summarized below. In terms of testing current views of feedback, recall that results from the Mory (1991) study indicated that predictions from the Kulhavy and Stock (1989) model held for verbal information learning, but not for concept acquisition. Swindell (1991) also reported a study attempting to examine the same model (Kulhavy & Stock, 1989) under the conditions of higher-level learning. Although results of the study claim to suggest the generalizability of the model to higher learning, questions required recall of verbal information only, with no guarantee that intellectual skill learning had occurred.

The vast majority of feedback studies have dealt with verbal information tasks (Schimmel, 1988). Consequently, it is not known if certain patterns or inconsistencies that have emerged from these studies would necessarily result when involving other types of learning. This question * has been acknowledged by a few researchers, an example of which is clear in Andre and Thieman's (1988) statement: "Whether feedback on questions facilitates concept learning as well as factual learning is not known from available research" (p. 297). Indeed, chimmel discovered differ ences in the value of informative feedback for declarative knowledge versus procedural learning in the results of a 1983 meta-analysis.

Smith and Ragan (1993) estimate feedback requirements for different learning outcomes based on the theoretical cognitive processing requirements of each outcome. Thus their suggestions are predominantly theory based, and the reader should note that each area is a source of much-needed research to test these conjectures. The following sections address the feedback requirements suggested by either research or theory, or both.

32.5.4.1. Learning Outcome Comparisons. In an effort to bridge the gap between learning outcome differences, some researchers have compared declarative information tasks with higher cognitive tasks. Lee (1985) compared verbal information with rule using, hypothesizing that feedback for rule-using tasks should be more complex than feedback for learning verbal information. Three levels of feedback were compared. Correct-answer feedback was the same for all three levels (i.e., "right"). Differences in feedback only occurred if the student missed the question. For an error, students in the first level of feedback simply received the statement "Wrong." Students in the second level were told "Wrong. The answer is ….for errors made. Errors for the students in the third level of feedback were presented with "Wrong. The rule is.... The correct answer is …. There were no significant differences between feedback levels, suggesting that more-complex feedback did not prove more effective in either task. An additional finding was that there were no differences between feedback that was given immediately or feedback that was delayed.

Another study comparing verbal information with rule using was completed by Char (1978). Char refers to his intellectual skill task as "higher-order learning," which he describes as both identifying concepts and applying rules. The purpose was to examine the effects of both informative feedback versus no feedback, and delayed versus immediate feedback on retention of verbal information and higherorder learning. As one might predict, informative feedback did significantly enhance retention of both verbal information and higher-order learning. There were no differences between immediate and delayed feedback. It is regrettable that he did not categorize each higher-order question separately as being either a concept or rule application, so as to more clearly delineate between the specific kinds of learning being applied.

S. U. Wager (1983) also compared verbal information learning with a type of intellectual skill-specifically, defined concepts. She examined the effects of timing and type Of feedback on retention of an instructional task involving verbal information and defined concepts learning. Both immediate- and delayed-feedback timing were used, and feedback was either simple or elaborated. Simple feedback presented a knowledge of results only, and elaborated feedback presented a combination of knowledge of results, knowledge of correct response, and response contingent feedback, which explained why a particular response choice was correct or not. Results indicated that neither timing of feedback nor type of feedback made any significant differences between groups. These results were partially attributed to the fact that the feedback may have assumed a lesser role when students were given tutorial instruction.

Gaynor (1981) also compared across verbal tasks and higher-level tasks. Rather than using Gagn6's categorizations of "verbal information" and "intellectual skill," Gaynor classified her materials according to Bloom's taxonomy. She compared test items that fell into three levels of intellectual ability: knowledge, comprehension, and application. She concluded that when degree of original learning is equated, immediate feedback, end-of-session feedback, or even no feedback have little effect on short- or long-term retention of materials at Bloom's first three taxonomy levels.

Mory (1991, 1994) attempted to test the Kulhavy and Stock (1989) model of response certitude using two different types of learning outcomes for her subjects to try to determine if the model would generalize to a conceptlearning task. The model was derived from studies that used predominantly verbal information and rote memorization of facts. In the Mory (1991, 1994) study, feedback was adaptive based on a combined assessment of answer correctness and level of certitude. The rationale was that by varying the type and amount of information contained in the feedback to fit the prescriptive state of learners under high- and low-certitude conditions and correct and error responses, learners would be given only the most "economic" form of feedback. Further, this type of adaptive feedback treatment was compared with a traditional form of nonadaptive feedback that essentially contained a verification component combined with knowledge of correct response. While there were no significant differences in posttest performance between the adaptive and nonadaptive groups, there was a significant increase in feedback efficiency for the adaptive group. Mory postulates that one reason that adaptive feedback did not seem to improve scores in the higher-level learning task of concept learning was that students did not accurately predict their answer correctness and thus were not able to receive the appropriate feedback for that condition. Data in the study revealed that certitude levels tended to be high throughout the adaptive program, regardless of actual answer corTectness. This means that students did not receive low-certitude feedback when needed most. Learners simply could not give accurate assessments of their own abilities to classify a particular concept. As stated earlier, these findings are supported by previous studies involving "feeling of knowing" judgments (which are similar to response-certitude estimates) that propose that when learning involved higher-level tasks, judgments tended to be overestimated by learners (Metcalfe, 1986). In contrast to this, some researchers have found that students learning concepts tended to underestimate their belief about their answer correctness (M. P. Driscoll, personal communication, Aug. 30, 1990). Despite the opposing nature of these two separate results, it would appear that learners do not accurately predict their knowledge in higher-cognitive tasks.

32.5.4.2. Declarative Knowledge. This type of knowledge is what is referred to as verbal information in Gagn6's (1985) taxonomy (see 18.3) and specifically by Smith and Ragan (1993) as including labels, facts, lists, and organized discourse. For labels and facts, feedback should give some evaluation of whether the leamer's response is complete and whether the learner's associations are complete. Lists will possibly involve the elements of both completeness and sequence to be evaluated. They suggest that feedback might point out errors in incorrect combinations of associations and that simple correct/incorrect feedback may be sufficient. In Schimmel's work (1983), confirmation feedback was found to be more potent than correct-answer feedback in verbal information tasks. Simpler feedback was more effective than complex feedback in a study by Siegel and Misselt (1984). Further, Kulhavy and his colleagues (1985) found that knowledge of correct response was more beneficial than more complex feedback,

In terms of organized discourse, they (Smith & Ragan, 1993) assert that feedback must act as an intelligent evaluator or provide model responses. This "intelligent" evaluation may be provided by a knowledgeable human being or by computerized intelligent tutors. In terms of a model response, feedback should be constructed with attention to modeling organization, links of information, and elaborations that would be considered essential for an appropriate answer.

32.5.4.3. Concept Learning. Four feedback studies were found which dealt specifically with concept-leaming tasks. Although already described under the feedback elaboration research, they will be discussed in this section for their importance as involving concepts. But before discussing these studies, an overview of concepts is presented from the major tenets of concepts-leaming research.

Concepts are types of classifying rules (Gagn6 & Driscoll, 1988; Gagn6, Briggs & Wager, 1992) that are used to facilitate the classification of instances through acquiring definitions, attributes, and examples (Tessmer, Wilson & Driscoll, 1990). The two categories of concepts are concrete concepts and defined concepts (Gagn6 & Driscoll, 1988). Concrete concepts represent categories determined on the basis of perceptual features, whereas defined concepts represent semantic categories that may or may not have a perceptual basis (Tessmer et al., 1990). Defined concepts must be identified through the use of a definition, rather than by actual sight.

Concepts have both declarative and procedural components that require instruction designed to convey both of these learning outcomes. Declarative strategies help make information about the concept meaningful to the leamer, and procedural strategies produce accuracy and ease in performance of concept classification skills (Tessmer et al., 1990). Conceptual knowledge is more than just the storage of declarative (or verbal information) knowledge, embodying also an understanding of a concept's operational structure within itself and between associated concepts (Park & R. D. Tennyson, 1986; R. D. Tennyson & Cocchiarella, 1986). Since conceptual knowledge is the storage and integration of information, and procedural knowledge is the retrieval of knowledge in the service of solving problems, instruction could typically include portions that focus on verbal information outcomes (the declarative component) and intellectual skill (concept) outcomes (the procedural component). Although testing how well a student has stored information in the form of verbal information outcomes is not a guarantee that the student also understands and can integrate the information, it still is an indicator of how much he or she can remember in order to apply it.

The primary method of teaching concepts usually involves presenting a definition or classification rule, followed by sets of examples and nonexamples. Examples and nonexamples are in the form of both (1) statement presentations to the student (expository instances) and (2) question presentations to the student (interrogatory instances) (R. D. Tennyson & Cocchiarella, 1986). Additionally, critical attributes of a concept may be presented. Critical attributes are what define a concept and must be present in any given case to be an example of the concept. The presence of these critical attributes constitutes both "necessary and sufficient conditions for judging the presence of the concept" (Wilson, 1986, p. 16). The test of whether a concept has been learned is to present the student with new instances of the concept that he has not previously encountered to see if he can classify the instance correctly.

Further, a concept is a set of specific objects, symbols, or events that share common characteristics (critical attributes) and can be categorized by a particular name or symbol (R. D. Tennyson & Park, 1980). Most concepts do not exist in isolation but as pall of a set of related concepts. The placement of a given concept in relation to other concepts having similar attributes implies that certain concepts would be subordinate while others would be superordinate. Those concepts that are placed in the same general location in the content structure and are neither subordinate or superordinate may be defined as coordinate concepts (M. Merrill & R. Tennyson, 1977; R. D. Tennyson & Park, 1980). Coordinate concepts fall at the same level of specificity, and the members of any coordinate class are not members of any other coordinate class (Klausmeier, 1976). For coordinate concept learning, the nonexamples of one concept are examples of other coordinate concepts. Usually a set of concepts are presented simultaneously, making it easy for the learner to confuse specific attributes of one concept with another one and resulting in an error of misclassification. But simultaneous presentation is helpful in enabling learners to compare and contrast similarities and differences between concepts and thus aid in clarification of individual concepts (Litchfield, 1987).

The first study to involve both feedback and concepts is by Waldrop and his colleagues (Waldrop et al., 1986). They approach feedback with an emphasis on feedback only being effective under certain conditions, relating the importance of this when using feedback in CAI. They compared three types of feedback during a drill-and-practice CAI program. The program presented a series of 20 examples of 4 types of consequences for behavior (positive reinforcement, negative reinforcement, punishment, and extinction). Although the classification of concepts was used in the practice, they did not test the learning of the concepts by giving them new instances on the posttest. Instead, the criterion measure consisted of the same 20 items used in the CAI modules, only presented in a random order and within a test booklet. At least in terms of retention of the original examples, immediate extended feedback following both correct and incorrect responses was superior to minimal feedback. It would have been of value if the researchers had tested the concepts in the manner typically in line with what theorists would say constitutes successful leaming of the concept-that is, being able to classify previously unencountered examples-and not merely by a repetition of the same examples.

A second feedback study to employ the use of concepts was by J. Merrill (1987). High- and low-level questions were used in combination with corrective feedback and attribute isolation feedback to form four versions of a computerbased science lesson that taught Xenograde terminology concepts. 1. Merrill chose attribute isolation feedback based on M. Merrill and R. Tennyson's (1977) proposition that the correct classification of newly encountered examples of a concept is more likely if attribution isolation is presented both in the instructional presentation of examples and in the feedback given after practice examples. The primary hypothesis of the study was that students who received high-level questions and attribute isolation feedback would perform better than the other groups. Although there was a question-level main effect of students in the high-level question treatments performing significantly better than those in low-level question treatments, there was an absence of a feedback form main effect. J. Merrill suggests that this absence may be due to the fact that potential benefits of either feedback form. were not fully available to the students. The attribute isolation feedback was only presented after two wrong responses and consequently was not encountered very often. This is unfortunate, considering that results from previous studies (cited in J. Merrill, 1987) yielded significant posttest results from the addition of attribute isolation to the concept-learning task.

Andre and Thieman (1988) approached the concept issue by directly addressing the problem that feedback research has used tests that measure only factual learning and thus has stood "mute on the issue of concept/principle acquisition" (p. 297). Unlike the Waldrop et al. (1986) study, these researchers measured both retention of the presented examples as well as performance on new instances of the concept. They broke student scores into performance on four types of questions: (1) repeated factual, (2) repeated application, (3) new factual, and (4) new application. Performance on the new application questions was cited as the main variable of interest, since the major purpose of the study was to determine the effects of type of question and type of feedback on concept learning. Subjects were given either factual, application, or both types of adjunct questions immediately after reading an instructional passage. A day later, subjects were given either (1) no feedback, (2) correct-answer feedback, or (3) self-correction feedback in which the students received a list of incorrect items without the correct answer, the instructional passage, and instructions to find the correct answers to the incorrect items.

One major finding of the study was that adjunct application questions significantly improved student performance on later use of concepts, and that this improvement occurred without any loss of incidental factual learning. This beneficial effect was obtained only when application questions were used in isolation. When both factual and application adjunct questions were used in the practice, poor performance occurred on new application items. This suggests some sort of interference when the two different types of questions are presented together.

A second major finding was that feedback did not influence concept learning (i.e., performance on new instances) but did influence performance on repeated examples of concepts. Thus, feedback did not facilitate the acquisition of a concept that could be applied to new examples. They suggest that more than one trial of feedback may have been insufficient to induce concept acquisition, and cited Park and R. D. Tennyson's (1980) finding that students required approximately four examples to learn a particular concept.

Dempsey and his associates (Dempsey, 1988; Deippsey, Driscoll & Litchfield, 1993) examined concepts in terms of achievement on a retention test, feedback study time, and type and numbers of discrimination errors. These studies examined the effects of four methods of immediate corrective feedback on retention, discrimination error, and feedback study time in computer-based instruction. Also, the studies explored the relationship between types of corrective feedback and the types of errors made by learners. The four feedback conditions included: (1) feedback that gave knowledge of correct response only, (2) feedback that informed students of the correct response and then required that they make that response, (3) feedback that gave knowledge of the correct response and also presented anticipated wrong-answer feedback, and (4) feedback that gave knowledge of correct response and allowed a second try to answer the question. No significant differences in retention rates resulted for any feedback group, but the group receiving knowledge of correct response only used significantly less feedback study time and was more efficient than the other conditions. Type of feedback made no difference in the number of errors during instruction. Students making fewer fine discrimination errors during the instruction performed better on a retention test. More fine than gross discrimination errors were made on the retention test. Regarding feedback study times and discrimination error, almost twice as much feedback study time was consumed for fine discrimination errors. This last finding may suggest a link between fine discrimination errors and high-certitude errors from Kulhavy's work, since in both cases, the longest feedback study times result.

32.5.4.4. Rule Learning. According to Smith and Ragan (1993), rules may be one of two types: relational rules and procedural rules. Relational rules involve relationships between two or more concepts, often being described in terms of "if-then" or "cause-effect" (p. 84). Relational rules have also been referred to as propositions, principles, laws, axioms, theorems, and postulates. These researchers (Smith & Ragan, 1993) describe suggested feedback for rule learning in terms of various practice stages for using the rule. When practicing verbalizing or visualizing the rule, feedback should provide information concerning the key concepts of the rule and their relationships. Note that this would basically qualify as verbal information and not rule utilization itself.

When practice involves the recognition of situations in which the rule is applicable, feedback should identify (1) whether the rule is applicable, and (2) what features of the situation make the rule applicable or not. They (Smith & Ragan, 1993) suggest that the explanatory portion of the feedback be placed under learner control, as explanatory feedback has been shown to confuse some learners (Phye,1979).

When learners begin actually applying the rule, feedback should provide the outcome of the application of the rule. Explanatory feedback might include a step-by-step solution of the problem, highlighting critical features that influence the application of the rule or illustrating in graphic form how a solution can be drawn. Such explanatory feedback was found to be significantly superior than simple correct/ incorrect feedback on college students' ability to apply rules in computer programming (Lee, Smith & Savenye, 1991).

When learners determine whether a ride has been correctly applied, feedback should include simple correctanswer feedback. For situations in which the rule has been applied incorrectly, feedback should point out the specific error in application and give the correct way that rule should have been applied. Feedback might also serve to provide hints for modification of the learner's use of a rule or be adapted to correct specific misconceptions or error patterns that a learner is making (Smith & Ragan, 1993).

The second type of rule, procedural rules, involves learning a series of steps to reach a specific goal. Procedural rules may be simple, with only one set of steps to complete linearly; or they may be complex, with many decision points leading to different paths or branches. Ile first step in learning procedural rules involves determining if the procedure is required. Smith and Ragan (1993) recommend feedback that is confirmatory, informing the learner whether he or she has appropriately identified the situations that require the application of the procedure. Learners should also be given feedback about the accuracy of their completion of each step in the procedure. During initial practice stages, feedback should be detailed and given during the practice of each step of the procedure. Then, as the learner is able to perform the entire procedure, feedback would both determine whether each step was correctly completed and provide qualitative information concerning selection, criterion, and precision and efficiency. These researchers (Smith & Ragan, 1993) also recommend that feedback be given about the remembrance of steps in the procedure and their correct sequence of completion. And finally, feedback should be provided about the appropriateness of a completed procedure in the form of correct-answer feedback.

Departing from the usual fare of verbal-learning studies in the feedback elaboration research, only a few experimenters have chosen to look at rule using alone. Birenbaum and Tatsuoka's (1987) study examined the seriousness of errors committed by eighth-graders using rules to add signed numbers in a CAI task. For serious errors, it did not matter how much elaboration was in the feedback; correction was relatively unaffected by feedback. Feedback elaborations for nonserious errors did have an increasing probability of being corrected as more information was added to the feedback.

A second group of researchers (Tait et al., 1973) examined rule using in a CAI environment designed to help children multiply two- and three-digit numbers by one-digit numbers. Treatment conditions included (1) no feedback, (2) passive feedback, and (3) active feedback. The active feedback procedure required an overt response to be given for each step in the procedure for computing the answer. The passive procedure merely printed a message to the student and required no overt response. The active feedback was designed to alleviate the problem of children not attending to feedback messages that explained the procedure. Children seemed to be copying the answer presented at the end of the feedback and ignoring other information in the feedback. Active feedback required the student's active engagement with the feedback at each step within solving the problem. Additionally, active feedback contained more information than did passive feedback.

Even when using both active and passive feedback, there was still little improvement from pretest to posttest. The researchers concluded then that with the active feedback, children were still able to copy answers without understanding the procedure behind diem. Consequently, a second experiment was designed which required the pupils to repeat the question until it had been answered correctly. The correct answer was required in both passive and active feedback groups before the child was allowed to continue on to a new problem. Even under these conditions, active feedback was no more beneficial than passive feedback. However, pupils who had scored low on the pretest did perform much better on the posttest when given active feedback than similar pupils in the passive feedback group.

32.5.4.5. Problem Solving. In the domain of problem solving, a learner must select and combine multiple rules in order to reach a solution. This may require that learners use declarative knowledge and cognitive strategies within a content domain, and combine previously learned relational and procedural rules to solve a previously unencountered problem (Gagn6, 1985). According to Smith and Ragan (1993, p. 92), the following stages often occur during a problem-solving task, and not necessarily in the same sequence:

1. Clarify the given state, including any obstacles or constraints.
2. Clarify the goal state, including criteria for knowing when the goal is reached.
3. Search for relevant prior knowledge of declarative, rule, or cognitive strategies that will aid in solution.
4. Decompose problems into subproblems with subgoals.
5. Determine a sequence for attacking subproblems.
6. Consider possible solution paths to each subproblem using related prior knowledge.
7. Select solution path and apply production knowledge (rules) in appropriate order.
8. Evaluate to determine if goal is achieved. If not, revise by returning to (1) above.

Since this type of learning involves the use of several other types of learning, feedback during a problem-solving task must work to help the learner see where his or her strategies or information gaps are occurring. According to Smith and Ragan's (1993) suggestions, initial feedback may be in the form of hints or guiding questions. It may include information as to which information has been used or misused, the appropriateness of selected solutions, whether individual phases of the solution have been correctly performed, and the efficiency of the solution process. As learners progress from novice to expert, their approaches to a problem should become more automatic. At this expert level, learners will need feedback on the efficiency or speed of their problem solving. The extent of this type of feedback will depend on the extent that genuine expertise is an expected part of the learning goal.

In simulations, feedback is often provided in terms of presenting learners with the consequences of their decisions. Open-ended response questions may be followed by feedback presenting a model of the solution process. And during the initial stages of practice, immediate feedback will be most helpful for intermediate stages, when responses can keep the learner from an eventual successful solution (cited in Smith & Ragan, 1993).

It should be noted that more recent views of problem solving are found in the literature on constructivism, to be presented later in this chapter. In particular, recent research in the areas of anchored instruction, situated cognition, situated learning, and generative learning have examined what might be thought to be "problem solving," but with very different philosophical assumptions about the way that learning takes place (Cognition & Technology Group at Vanderbilt [CTGVI, 1990, 1991a, 1991b, 1992a, 1992b, Young, 1993). It is from this broadened per-spective that researchers will find the most need for research on types of feedback that can aid learners as they construct solutions to authentic problems.

32-5.4.6. Cognitive Strategies. Cognitive strategies are techniques that learners use to help them attend to, organize, elaborate, manipulate, and retrieve knowl controlling their own cognitive processes (s 1985). Smith and Ragan (1993) relate the use ol strategies with oroblern solving since the selection, application, and evaluation of a cognitive strategy is sinidlar to problem-solving techniques. Given that siniila~ty, feedback will have some of the same functions as ~tated for problem solving-that of modeling appropriate decisions and stating explicitly whether the decisions and mance of the learner were adequate or not. Feedback should also contain explanations as to why the appropriate. Characteristics such as the leamers'ca requirements of the task, learner efficacy, and applications of various strategies should be considered as W (Smith & Ragan, 1993) suggest that for open-en, toward a solution, feedback should involve reviewing appropriateness of a particular strategy and critical details of the strategy for a given problem/solution.

In a study by Ahmad (1988), college-age learners participating in a guided discovery lesson were taught strategies that were either compatible or incongruent with th cognitive strategies. When feedback on the e ineffective use of a particular strategy was provid, better performance resulted when the strategy was compatible with previously employed strategies. But when the, strategy used by the learner was incompatible with her or ;.,s prior strategy use, feedback containing only whether or not a solution was correct or incorrect proved more effective.

Since cognitive strategies can be very subject domain oriented, it would probably be fruitful to explore tuhT uses of various cognitive strategies within specified subj~'ct areas and contexts. Also, as stated above, researcher~ should consider examining cognitive strategies in terms of their applications to a learner's construction of solutions Of more authentic learning tasks. In fact, one of the goals un the development of the Jasper series (CTGV, 1990 1992a, 1992b) is the importance of helping stude to become independent thinkers, learning to idenify and define issues and problems on their own (CTGV, 11992a). The whole notion of cognitive strategies should be viewed as learners themselves generate the slihn-hlems and data necess- to satisfv subgoals that they themselves have generated on their own, refenleld to as "generative learning" (CTGV, 1990, 1992a).

32.5A.7. Psychomotor Skills. Psychomotor 1~utrning involves skills that are physical in nature, often with coordinated muscular movements. Psychomotor skills require a cognitive component, particularly in the early stages of learning the skill. As the skill becomes more automatic, the cognitive awareness becomes an unconscious part !,of performing the skill. Two components of psychomotor skill are (1) executive subroutines to control decisions and! supply subordinate hierarchical skins and (2) temporal pa4eriiing of skills to integrate the sequence of performanoe over time, involving pacing and anticipation (cited in Sinith & Ragan, 1993). Further, psychomotor skills are sometimes classified on a continuum from "closed" to "open." Closed skills are predictable and do not require much adaptation to the environment, thus referred to as "internally paced" (Singer, as cited in Smith & Ragan, 1993). Open skills, on the other hand, must be adapted to unpredictable aspects of a changing environment.

The function of feedback in the learning of psychomotor skills is to provide a surrogate for the learner of self-evaluation, at least until the learner reaches a skill level where he or she can provide this role for themselves. However, as Snidth and Ragan (1993) point out, this transfer is more pronounced than in other types of learning tasks. Learners are able, through their own seeing and hearing, to deterrmine when a skill has been performed correctly, thus providing themselves a type of internal feedback.

Feedback may be given about (1) the product (the quality of the response outcome) or (2) the process (what causes the response outcome). During the beginning practice stages of motor skill, feedback serves the critical function of providing information about the process of executing the motor skill. Then, as a learner advances in his or her ability to execute the skill, feedback can focus on the response outcome (product) itself. Ho and Shea (cited in Smith & Ragan, 1993) found that learners appeared to learn simple motor skills better when feedback was withdrawn or at least not given after every single response. Also, quantitative feedback (using a measurable criterion) appears to be superior to qualitative feedback (e.g., "too fast," "too low") (Smoll, as cited in Smith & Ragan, 1993). However, there is an optimal precision point to include in feedback, past which point can result in detrimental learning (Rogers, as cited in Smith & Ragan, 1993).

Graphic representations can be very beneficial to learners when included in feedback about the quality of a psychomotor response. Sometimes referred to as "kinematic" feedback, it can increase both efficiency and effectiveness of the learner during the acquisition of a psychomotor skill. Further, feedback that is interspersed throughout the learning of a motor task is more effective than massed feedback at the end of practice (cited in Smith & Ragan, 1993).

32.5.4.8. Attitude Learning. The final type of learning capability that will be discussed in this section is that of attitude learning (see 34.2). The desired outcome of attitude learning is that a learner will choose to behave in a particular way. A person's attitude about something is reflected in the decisions or choices he or she makes. The goal of instruction for attitude learning would be to influence what a learner chooses to do after the instruction is completed (Gagn6, 1985; Gagn6 et al., 1992). Obviously before a person can "choose" to do something, there are cognitive and behavioral components that have to be learned beforehand. The person has to cognitively "know how" to practice the attitude. Also, a person has to see the need to apply the attitude, behaviorally responding to opportunities to make decisions and make the particular choice. This can be accomplished through his or her own experience or vicariously through others' experiences. The affective side of attitude learning merely involves "knowing why."

Feedback for the cognitive and behavioral components can simply include information concerning whether they have successfully employed the knowledge or skill that the attitude will require. Feedback can also include information about the congruency of their responses with the desired attitude. In terms of mediating attitudds through feedback, learners can be presented with information concerning the anticipated consequences of their choices, incorporating the affective component of why the beha~ior that reflects the attitude is important (Smith & Ragan, 1,993).

When one begins to speak of motivation in feedback, it is easy to bring to mind the reinfbrciemen~ view of feedback, and indeed, theories of motivation have tended to focus on behavioral reinforcement and performance rather than on increasing motivation through instructional means (Jacobs & Dempsey, 1993). In order to understand ways in which feedback can be used to help the motivational level of students, whether from a behavioral or a cognitive view, it will be useful to examine briefly some basic theories of motivation that psychologists have constructed to explain motivation in the learning process.

32.5.5.1. Goals and Goal Disc pancy Feedback. Past research in the area of motivation cited in Covington & Omelich, 1984) has shown that for a learner to remain motivated and involved depends on a close match between a learner's aspirations or goals and his or her expectations that these goals can be met. If these aspirations are set so high that they are unattainable, the learner will likely experience failure and discouragement. Conversely, when goals are set so low that their attainment is certain, success loses its potency in promoting further effort (Birney, Burdick & Teevan, 1969). Covington and Om~lich (1984) have suggested that setting performance goals beyond present capabilities, particularly in the case of low self-perception of success, can become a main sourc~ of gratification. Apparently the statement of a worthy 'goal is enough to boost self-regard irrespective of goal attonment. One might say that feedback is a means to allow a learner to study and "retest" information, actions that, according to some researchers, would encourage greater performance aspirations coupled with increased confidence to achieve these elevated goals. Findings suggest that motivation is a key mediating factor in the performance of le~arners (Covington & Omelich, 1984).

Feedback can be a powerful motivator when it is given in response to goal-driven efforts. Somo researchers suggest that the learner's goal orientation should be considered when designing instruction, particularly When feedback can encourage or discourage a learner's effort, thus regulating. sustained effort and future goal orientations (Dempsey, Driscoll & Swindell, 1993). Other researchers claim that feedback enters into the actual goal-setOng process, as a basis for evaluating assigned goals and in guiding the formation of a learner's personal goals! (Erez & Zidon, 1984; Locke, Shaw, Saari & Latham, 198 1). Malone (198 1) asserts that there are certain attributes that a goal must have in order to challenge the learner to attain them. First, they should be personally meaningful and easily generated by the learner. This is supported by Locke and others who contend that goals may enhance performance only when the learner conscientiously accepts them (Locke et al., 1981). indeed, Erez and Zidon (1984) found a linear decrease in performance after assigned goals were rejected.

Malone (1981) also suggests that learners need some type of performance feedback as to whether or not they are achieving their goals. This notion was explored in a study by Vance and Coella (1990) in which goal discrepancy feedback (GDF) and past-performance discrepancy feedback (PDF) were used to examine acceptance of assigned goals and personal goal levels of learners. GDF conveyed to what level learners were performing above or below the assigned goals. PDF indicated the learner's performance level from one trial to the next. Interestingly, assigned goals were designed to become increasingly difficult over given trials. This meant that, concurrently, the GDF became increasingly negative and, consequently, the learner's acceptance of the goals because less likely. Learners were found to switch over to PDF for evaluating assigned goals and for selecting new goals, what one would expect given the uncomfortable nature of the GDF over time.

Hoska (1993) refers to goals in terms of whether they help in acquiring something desirable or in avoiding some thing undesirable. These acquisition and avoidance goals can be external (in which the learner's focus is performing for others) or internal (in which the learner's focus is on learn ' ing for oneself). Several researchers (Dweck, 1986; Dweck & Legget, 1988; Nolen-Hoeksema, Seligman & Girgus, 1986) have found that an individual's general goal orientation falls on a continuum between an ego-involved performing-goal orientation to a ask-involved learning goal orientation. She further explains that learners who have performing goals want to demonstrate high ability and to avoid poor performance. They tend to view their success as a display of their abilities, which they measure in terms of the perceived abilities of others. To an ego-involved learner, ability is his key to success, and effort is merely a means to achieve such external goals. In contrast, individuals who have learning goals pursue learning and extend effort to gain skills. They view their competence as improved mastery, attained through effort. To a task-involved learner, effort is perceived as being beneficial since it helps the learner attain mastery.

When learners are successful, individual goal orientation is not a critical issue since success breeds the desire to extend effort, regardless of the goal. But when looking at instances of performance failure, the two goal orientations can produce very different results. If an individual with a leaming-goal orientation perceives an impending failure, it results in his or her exerting more effort to the task. To this task-focused individual, obstacles are a challenge to be 'Overcome through effort. Task-involved learners believe that effort, not ability, is the key to success and, consequently, they will look for ways to overcome any difficulties that arise. Their satisfaction lies in effort, which has been shown to result in higher mastery scores and produces 50% more work than with other learners (Dweck, 1986).

In contrast, learners with a performance goal orientation will react quite differently to an impending failure. Obstacles become a threat to success and, therefore, a threat to their self-worth. Even high-ability learners in this group will set up defenses to protect themselves against the emotional threat. These self-defense reactions include such tactics as discounting (Kelley, 1973); avoiding the task, feigning boredom, or engaging in task-irrelevant actions to bolster their self-image (Dweck & Legget, 1988); and using inefficient strategies, resulting in learned helplessness (Seligman, Maier & Geer, 1968).

According to Hoska. (1993), if learners begin a task without a predisposition toward one of these two goal orientations, they will probably approach the task with both the goals of learning and performing. If learners do not receive cues favoring one type of goal over another, they will act according to their predisposition. But if a learning situation is structured to foster a particular type of goal, learners will respond. Thus a learner's goal orientations can be temporarily and, over time, permanently altered by intervention. This is where feedback can have a great effect on this aspect of motivation.

Providing lesson feedback can be used to influence a learner's goal orientation by increasing her or his incentives to learn and minimize a learner's incentives to perform. Hoska (1993) classifies these modifications into three approaches: (1) changing the learner's view of intelligence, (2) modifying the goal structure of the learning task, and (3) controlling the delivery of learning rewards. In terms of modifying a learner's view of intelligence, feedback can help learners view intelligence in a way that helps them see that ability and skill can be developed through practice, that effort is critical to increasing this skill, and that mistakes are part of the skill-developing process.

In terms of altering a learner's goal structure, one should consider the type of learning environment within which the lesson is taking place. Often goal 'structures are set within competitive, cooperative, and individualistic learning environments. Competitive goal structures emphasize performance success and failure and cause learners to become ego involved. Cooperative goal structures teach a learner that the task is important, thus helping to foster learning goals (Johnson & Johnson, 1993). In individualized goal structures, although noncompetitive, a learner will not necessarily be task focused, but the learner's orientation will be determined by the reward system of the learning experience.

Lastly, the control of the delivery of learning awards usually involves providing external awards, offering praise and blame feedback, and offering unrequested help that can increase the learner's chance for success, and the comparison of the learner's performance to that of others.

Unfortunately, providing external rewards to learners can easily undermine any personal learning goals that a learner has. Researchers have found that learners will often select less-difficult tasks to increase their probability of success (Decil 1972; McCullers, Fabes & Moran, 1987), and this effect increases under competitive conditions (Covington & Omelich, 1979). Further, learners often think that only difficult or boring tasks require reward (McCullers et al., 1987). Hoska (1993) offers the suggestion that feedback on the development of skills at various stages of a learning task can help redirect the learner to a focus on internal rewards.

Praise and blame feedback, once thought to provide positive and negative reinforcement, has been shown to be interpreted by learners as estimates of their ability (Deci, 1972). While most learners associate praise and blame in terms of how much effort they expended, ego-involved learners and learners in competitive tasks often interpret praise-and-blame feedback as indicators of both ability and success levels, sometimes even producing learned helplessness (Koestner, Zuckerman & Koestner, 1987). Hoska (1993) summarizes the effects of praise-and-blame feedback in terms of whether or not the learner felt the comments were warranted, the difficulty of the task involved, and the goal structure of the learning environment. She points out that praise has the most potential for being misinterpreted by learners. When high praise occurs after successful completion of an easy task, it is interpreted to mean that the evaluator thinks the learner must have low ability. When minimal praise occurs after the successful completion of a difficult task, learners may believe that the evaluator believes they have high ability, with success occurring due to this high ability rather than effort. And when praise or no feedback occurs after a failure, learners will tend to believe that this indicates low ability.

Blame feedback for incorrect responses can have more positive effects than praise feedback does for successes. Learners will tend to perceive blame as a result of their withheld effort. Hoska (1993) cautions that blame feedback must be used carefully since it also can be harmful in instances when a learner has invested a high degree of effort and has achieved at least some level of success. In such cases, the feedback can teach learners that small, sustained improvements do not help them reach masteryan undesirable outcome. In general, praise-and-blame feedback should focus on individual learner responses rather than on overall success levels so as to associate the feedback with effort and not with ability.

It should be noted that having the option of being retested, in which a learner is given feedback and allowed to improve, also increases the number of failures experienced by a learner (Covington & Omelich, 1982). These failures have been shown to lead to decreases in self-estimates of ability, which in turn trigger hopelessness, shame, and anxiety (Covington, 1983; Covington & Omelich, 1981). But under a mastery format, positive perceptions of ability have been shown to be maintained even in the event of failure as long as learners eventually reached their grade goals or showed improvement (Covington & Onielich, 1984). In the same study (Covington & Omelich, 1084), while isolated failures were temporarily demoralizing, they were shown to play little part in determining overall motivational reaetions. When students do not have opportO,nity to make good their failures, the result is greater student demoralization even though students experience fewer failures. The study makes the point that task-oriented learning may be especially beneficial for slow learners who may require several tries before mastering the subject matter.

Although the mastery learning approach is not new, nor is the idea of mastery being a desirable'! approach for slow learners, it is important to note here th~t the motivational element at work in such approaches should not be ignored. This line of motivation research suggests that students who are given the chance to improve through practice and feedback of some sort will have a positive perception of ability and will retain a high level of rplotivation overall. Thus the "retesting" effects of feedbackilhave implications for improving and sustaining motivation, ~lirrespective of the numbers of errors made.

32.5.5.2. Self-Efficacy and Expectancy. Self-efficaey and task expectancy have been said to b6i equally as important in determining how a learner will respond to a learning task (Hoska, 1993). Self-efficacy is the learner's perception of how well he or she can perform the i learning tasks to achieve his or her goals. It helps the learner select attainable goals and determine the amount of e4ort that will be involved for reaching success. Self-efficaq affects learning because it influences how much effort a learner will invest in a task. For example, low self-efficacy can cause learners to dwell on their deficiencies, resulting in inaccurate personal assessments of task difficulty and ex cessive attention devoted to the possibility of failure, resulting in a learning detriment (Bandura, cited in Hoska, 1993). On the other hand, high self-efficacy does not always result in maximum effort, because the amount of effort extended by learners is said to depend on not only self-efficacy but also on goal incentives and the perceived demand or load of a task. Hoska (1993) points out that when learner$ are aware that a task is demanding, high self-efficacy will usually result in the effort needed for optimal performance. But when learners perceive tasks as being easy, high self-efficacy may cause them to feel that minimal effort is needed.

Bandura (1977) cites three information sources from which learners derive their general sense', of self-efficacy. One is through vicarious experiences, in which self-efficacy is increased through viewing others' succes~es, or decreased when viewing others' failures. Self-efficapy is also developed through the learner's own personal p 01rformance. Tho impact of a success or failure affects self-efficacy by how the learner interprets the outcome. Any isuccess that is achieved through a minimal amount of effort is viewed to indicate high ability and can result in increased self-efficaey. Some learners view success that requires high effort to mean low ability, thus reducing self-efficacy of those learners. The third area that learners build their self-efficacy from is verbal persuasion. Verbal persuasion comes in the form of opinions from parents, teachers, and peers concerning the learner's ability to perform various tasks and tend to affect learners' own perceptions about their abilities. Even learners with an initially high level of self-efficacy are said to have their own opinions of their ability affected by continual exposure to negative criticism (Hoska, 1993). Self-efficacy levels can also be temporarily affected by the learner's physiological state (Bandura, 1977), role assignment, familiarity with a task, or presence of a highly confident person (Bandura, 1982).

Expectancy is determined by the amount of effort a learner deems as appropriate for a task, based on the learner's goal incentives. Hoska (1993, p. 119) lists several elements of expectancy as follows:

• Belief that an outcome, or goal, is possible given the current situation. Oxarners must feel that they have some control over goal attainment; this goal may or may not be task completion.)
• Belief that an outcome, which can be achieving either an acquisition or an avoidance goal, will have desired consequences. (The consequences of goal achievement must have some value to the learner.)
• Determination of the amount of effort appropriate for goal attainment. (The greater the goal incentive, the more effort the learner is willing to invest to achieve the goal.)
• Determination of whether or not the selected amount of effort will lead to goal attainment.

Keller and Suzuki (1986) assert that learners tend to evaluate outcomes against their own expectations. Recall that Kulhavy's research in the area of response certitude gives support for the importance of learner's expectancy level. Dempsey and his colleagues (Dempsey, Driscoll & Swindell, 1993) note that Kulhavy's work supports the hypothesis that "corrective feedback should be personally relevant to the learner and tailored to the learner's expectancy for success" (p. 28) and that this link has major implications for both motivational and instructional designs.

Hoska (1993) asserts that self-efficacy and expectancy levels can be modified. Figure 32-5 depicts the relationship between a learner's goals and self-efficacy with level of effort and task expectancy.

As can be seen in Figure 32-5, a learner's self-efficacy and strength of task goals influence the level of effort that the learner will decide to invest in the task. This selected level of effort will then affect the learner's task expectancy, which will in turn influence further effort decisions. A learner's level of effort can be increased by providing him with experiences that are positive and internally satisfying, such as experiencing continually increasing levels of competence. Another method of increasing self-efficacy is by modifying the learner's attributes of success and failures (see the following section on attribution theory).

32.5-5.3. Attribution Theory. One classic approach to motivation emphasizes die importance of causal attributions in explaining the consequences of academic failure and success (Weiner, 1972, 1979, 1980). According to attribution theory, a learner's striving for achievement, affective reactions, and expectations concerning future outcomes are determined in part by the learner's attributional conclusions. Following performance on a leaming task, students will react in a generally positive or negative manner, formulate causes to explain their performance (causal attributions), and then experience affect and expectancy changes dependent on the nature of these attributions. Note how closely this last description matches what Kulhavy and his associates (Kulhavy, 1977; Kulhavy & Stock, 1989) described for a learner's processing of feedback and the comparison of his or her response to the feedback information. Recall that Kulhavy explained how a learner's level of response confidence combined with the actual correctness of response determined how feedback was used.

Forsyth and McMillan (1981) describe Weiner's proposed model of educational attributions and attempt to assess the relationship between the attributions, affect, and expectations of college students following a course exam. They cite previous research that suggests that when students attribute their success to factors such as ability or the nature of the task, their expectations for success increased, whereas students who attribute their success to luck or effort report less positive expectancies. Further, according to self-worth theory, "failure is more likely to lead to shame, depressed expectations, and lowered self-worth when it is ability linked rather than effort linked" (p.394). Effort is something that is within the learner's control and has been found to have a strong relationship to affect. In the Forsyth and McMillan (1981) study, the affective reactions of students who felt that their performance was caused by factors they could control were more positive than the reactions of students who believed they did not control the cause of their outcome. This supports studies of learned helplessness in that even students who did well on the test yet believed they could not control their outcomes reported less-positive affect.

Figure 32-5. Relationship between goals, self-efficacy, learner's selected level of effort, and task expectancy (Hoska, 1993, p. 121). (From Interactive Instruction and Feedback, p. 121, by J. V Dempsey & G. C. Sales, eds., 1993, Englewood Cliffs, NJ: Educational Technology.) Copyright 1993 by Educational Technology Publications. Reprinted with permission.

Learned helplessness has been described by Seligman as "the giving-up reaction, the quitting response that follows from the belief that whatever you do doesn't mattee, o9go, p. 15). In his 25 years of research in this area, Seligman has isolated what he believes to be "the great modulator of learned helplessness," that of explanatory style. Whenever events happen to a person, whether good or bad, each individual has a habitual manner in which he or she explains those events. These explanatory styles can either prevent helplessness or spread helplessness, depending on the person's explanations about the event. He further divides these explanations into the areas of permanence, pervasiveness, and personalization. He has found that if you can alter the way in which a pessimistic person explains a success or failure-that is, alter the levels of permanence, pervasiveness, and personalization they surround their self-talk with-you can change that person's outlook to one of optimism. Optimism, in turn, prevents the person from remaining in a state of helplessness so that he or she can be a more productive individual.

Since students' "perceived that noncontingency" (Forsyth & McMillan, 1981, p. 400) is associated with loss of achievement motivation, it seems reasonable to suggest that feedback could help students directly see a link between their level of effort and success, and provide information concerning various factors that the learner has under control. This will be elaborated on further in the next section, in which strategies for modifying learner's motivational perspectives are examined.

32.5.5.4. Modifying Learner's Perspectives Through Feedback. Hoska (1993) cites several steps that learners go through when they select and perform tasks, based on Weiner (1979). The steps are as follows. A learner:

1. Selects a goal.
2. Evaluates task difficulty.
3. Evaluates his or her abilities and develops a level of self-efficacy.
4. Selects an effort level and decides if that level will yield task success.
5. Invests effort to complete the task and evaluates
6. Determines and dimensions the cause of the success of failure.progress toward task completion.
7. Modifies his or her learner perspective.

As learners go through these steps, Hoska suggests feedback according to its motivational function. This is summarized in Table 32- 1.

32.5.5.5. ARCS Model of Motivation. Some researchers (Keller, 1983, 1987a, 1987b, 1987c; Keller & Kopp, 1987; Keller & Suzuki, 1987) have developed a model for increasing student motivation through instructional design, emphasizing instructional components that serve to motivate learners. The model grew from a macrotheory of motivation and instruction developed by Keller (1983). It is grounded in expectancy value theory that assumes that "people engage in an activity if it is perceived to be linked to the satisfaction of personal needs (the value aspect), and if there is a positive expectancy for success (the expectancy aspect)" (Keller, 1987a, pp. 2, 3). The model came about by dividing the value components into the categories of interest and relevance. Interest refers to attentional factors in the environment, and relevance refers more to goal-directed activities (p. 3). The expectancy component remained as a category, and a fourth category was added that was originally called ou4comes. Expectancy refers to one's own expectation for being successful, and outcomes refers to the reinforcing value of instruction. Outcomes include both reinforcement as described in operant-conditioning theory, but they also include any environmental outcomes that help maintain intrinsic motivation (see Deci, 1972).

The ARCS model was created by generating a large list of motivational strategy statements, derived from research findings and from practices that have resulted in motivated learners. The four original categories of 'interest, relevance, expectancy, and outcomes were renamed to strengthen the central feature of each component and to generate a useful acronym (Keller, 1987a). The model now focuses on the four categories: attention, relevance, confidence, and satisfaction, and is hence referred to as the ARCS model in reference to these areas. By using each of these four categories as a framework, instructional designers are able to incorporate strategies that relate to each.

When Keller (1987a) refers to attention, he is referring to the interest level of the learner-whether or not the learner's curiosity is aroused and is sustained over an appropriate period of time. Whether the learner perceives the instruction to satisfy personal needs or to help achieve personal goals is referred to by the relevance component of the model. Confidence refers to the learner's perceived likelihood of success (expectancy) and whether the learner perceives success as being under his or her control. Intrinsic and extrinsic motivation are referred to under the satisfaction component and focuses on the learner's intrinsic motivation and response to extrinsic awards.

Keller (1987c) notes that one of the challenges of motivation is that it is just as detrimental to leaming and performance for learners to be overmotivated as it is for them to be undermotivated. Undermotivation results in low productivity levels, while overmotivation results in high error rates and poor efficiency due to stress and overconfl dence (pp. 2, 3). The typical graphical representation of this is the inverted-U curve illustrating this result (see Fig. 32-6).

Keller (1987c) uses this inverted-U depiction when he completes audience analyses, plotting the levels of attention, relevance, confidence, and satisfaction on the curve. The rise and fall in performance in relationship to levels of motivation has implications for instruction. It appears that enhancing motivation for learning is an area that should be of concern to researchers, and, as we shall see momentarily, an area that feedback potentially may influence.

In Keller's (1983) original description of the motivational design of instruction, he lists several strategies to enhance motivation, many that are recommendations for the use of feedback to the learner. For our purposes of considering areas for future feedback research, these deserve closer inspection. They are as follows.

To enhance expectancy, what is now included in the model as confidence, "increase expectancy for success by using attributionalfeedback and other devices that help students connect success to personal effort and ability" (p. 420).

Attributional feedback is important when a student does not perceive a connection between his or her effort and its consequences. This is what has been referred to earlier as learned helplessness. A person who has developed learned helplessness towards a task does not perceive any causal link between behavior (effort) and its consequences. This type of learner cannot see the connection between ability and persistence as the key to success. When working with this type of learner, a sequence of problems or assignments should be developed that are initially easy but become challenging. After each success, feedback would be given as encouragement to keep trying, and after success at the more difficult problems, attributional feedback would be presented. Basically attributional feedback tells the learner that his or her success occurred because he or she kept trying. Keller (1983) refers to this feedback as being given verbally by a teacher in a classroom situation, but it is easily conceivable that adaptive feedback in other forms that contain the same type of messages would be appropriate.

To enhance the learner's perception of outcomes is now referred to as satisfaction and involves both intrinsic and extrinsic motivation. Keller (1983, pp. 426, 427) recommends to teachers to:

This first strategy is concerned with the types of consequences that will enhance or suppress intrinsic motivation. Keller (1983) points out that intrinsic motivation tends to flourish, more in a context of positive but noncontrolling consequences than when excessive evaluation and aversive forms of control are used (p. 426). In terms of motivating feedback in the second strategy, the behavioral view of operant conditioning using positive reinforcement again surfaces. As Keller emphasizes, we aile more likely to repeat behaviors that have pleasurable consequences than those that do not. When a learner receives positive reinforcement following a desired response, it affects the quantity of performance. One might contest this view of feedback in light of the evolution of feeO ' ack research from this type of behavioral view to that of cognition only. But it does make sense in terms of increasing and maintaining motivation or morale.

Figure 32-6. Inverted-U curve depiction of the relationship between motivation and performance. (Baso on Keller, 1987c.)

The last strategy refers to formative feedback, used to affect the quality of performance. It signals a gap between the given performance of the student versus the desired performance, and it indicates the actions to take to close the gap. Again, it is easy to see that this is feedback with the purpose to correct errors, as seen in the latest feedback studies that view feedback from a cognitive standpoint with a predominantly corrective function.

32.5.6.1. Paradligin Shifts. Ile majority of feedback studies in the literature have examined feedback under the traditional learning theory paradigms of I : behaviorism and information processing. Both of these theories can be classified as viewing learning from an objectivist perspective.

The philosophy of objectivism basically h6lds that "reliable knowledge about the world" exists (Jonassen, 1991b, p. 8) and that instruction serves to present this roal-world knowledge to the student who will in turn betested and "give back" this knowledge in order to demonstrate effective learning. Feedback would then serve to correct misinformation about this external, objective reality. This is, indeed, how most feedback studies are conceived.

The latest philosophy of learning, however, postulates that there is no external knowledge the student merely "takes in"; rather the student must construct his or her own reality or knowledge, and this construction will be based on the learner's prior experiences, mental structures, and beliefs (Brown, Collins & Duguid, 1988; Cooper, 1993; Duffy & Jonassen, 1991; Jonassen, 1991b). Put succinctly, "Knowledge is constructed in the mind of the learner" (Bodner, 1986, p. 873). This espouses the philosophy called constructivism, in which each learner constructs his or her own reality through interpretation of experiences of the external world (see 7. 1). And given this new view of learning, feedback will likely function differently than from an objectivist view of leaming (Mory, 1995).

Recall how early studies of feedback evolved from a behavioral view of feedback as reinforcement to more recent research that advocates an information-processing perspective with an emphasis on error correction. Feedback's main function is that of providing corrective information. Recall also the recently developed models of feedback (Bangert-Drowns et al., 1991; Kozma & Bangert-Drowns, 1987; Kulhavy, 1977; Kulhavy & Stock, 1989) that attempt to explain what happens within the feedback process. These models also contribute to an organization of the many variables that have been examined or even overlooked by past research. All of these studies were conceived under a philosophy of learning that embraces certain assumptions about learning from an objectivist viewpoint. These assumptions and the resulting use of feedback may be seen in Figure 32-7.

Although there has been progress in determining ways in which feedback can best be used under certain conditions, there are still many areas in which the feedback literature is not consistent, and yet other areas that have been left unexplored. One must critically examine feedback in light of the philosophical assumptions underlying these studies in order to highlight how feedback functions within such connived experimental settings. The basic assumptions of the objectivist philosophy are presented in order to contrast these assumptions with those of a consructivist view (Fig. 32-7). Suggestions for the use and function of feedback within the constructivist philosophy are 'presented in light of these basic assumptions in an effort to identify areas in need of further research (see Fig. 32-8).

Given such an array of inconsistencies in thefeedback literature, it is essential to question whether or not researchers are focusing on feedback variables that have real value for the world of the classroom. Many feedback studies are computer-based training (CBT) studies and are not intended to be generalized to a large group setting such as a "typical classroom." In most instructional~ settings, feedback is presented within some sort of intqractional environment, one that is not necessarily one of computerbased or programmed instruction. Perhaps soni,e of the most potent feedback is received within a setting !,in which the student interacts with some problem he or shei is trying to solve, with feedback resulting as a natural phenomenon of the context of instruction. For example, students who are trying to learn to play a musical instrument receive constant feedback from their mistakes just by hearing the sounds that are being produced, regardless of whether or pot there is any other external mechanism in place to correct these sounds. Feedback occurs as a natural result of interactions between the learner and his or her own constructions of knowledge. Further, the relevance of topics typically being presented within traditional feedback studies are usually a far cry from being anything the learner would be ml otivated to learn, this being purposefully the case in order to maximize feedback differences. The context in whichilearning takes place in most of these studies is often artificial and distanced from what a typical learner's interactions with a problem would be. Certainly the inconsistencie 9 in the feedback literature warrant some fresh ideas andl~perspectives. This researcher proposes that feedback be critically examined within a paradigm that embraces the philosophy of constructivism, in which the learner must cons~truct his or her own knowledge based on interactions within 41tudientic leaming environments (see 7.3, 12.3).

32.5.6.2. Applications of Feedback in Constroctivism. The philosophy of constructivism, opens a new avenue for feedback research.

Figure32-7. Assumptions of objectivism (from Jonassen, 1991b) and suggested use of feedback.

Figure32-8. Assumptions of constructivism (from Jonassen, 1991b) and suggested use of feedback.

Feedback in a constructivist context would provide intellectual tools and serve as an aid to help the learner construct his or her internal reality. Because learners would be solving complex problems through social negotiation between equal peers and within contextual settings, feedback might also occur in the form of discussion among learners and through comparisons of internally structured knowledge.

Perhaps to understand better what feedback would represent in a constructivist paradigm, consider the earlier transition of research foci from a behavioral view (reinforcement) to a cognitive view (information). As Cooper (1993, p. 16) suggests:

The move from behaviorism through cognitivism to constructivism represents shifts in emphasis away from an external view to an internal view. To the behaviorist, the internal processing is of no interest; to the cognitivist, the internal processing is only of importance to the extent to which it explains how external reality is understood. In contrast, the constructivist views the mind as a builder of symbols-the tools used to represent the knower's reality. External phenomena are meaningless except as the mind perceives them.

One constructivist principle is that instruction should occur in relevant contexts (Brown et al., 1989; Jonassen, 1991 a). Referred to as situated cognition, the notion is that learning occurs most effectively in context, and that the context becomes part of the actual knowledge base for that learning (Jonassen, 1991b). One approach to this is called cognitive apprenticeship (Brown et al., 1989; Collins, Brown & Newman, 1987; see 7.4, 20.3), in which learners engage in activity and make deliberate use of both social and physical context, just as an apprentice would do. Feedback in this view would occur in the form of the interactions between the learner and the activity of solving real-world problems. Rather than providing predetermined instructional sequences, feedback could be used as a coaching mechanism that analyzes strategies used to solve these problems (Jonassen, 1991b).

Another constructivist strategy has been termed cognitive flexibility theory and involves the presentation of multiple perspectives to learners (Jonassen, 1991b; Spiro, Feltovich, Jacobson & Coulson, 1991a, 1991b) (see 7.3, 23.4). By stressing conceptual interrelatedness, providing multiple representations of content, and emphasizing "case-based instruction" that includes inhere Int multiple themes (Jonassen, 1991b), feedback can help learners acquire advanced knowledge in ill-structured domains. Spiro and associates (Spiro et al., 1991a, 1991b) propose the use of m I ultidimensional and nonlinear hypertext (see 21.3) systems to convey ill-structured aspects of knowledge domains and thus promote cognitive, flexibility. When a learner approaches a problem from a certain perspective, feedback can serve to guide the learner to revisit the same material in a rearranged context, for a different purpose, from a different conceptual perspective (Spiro et al., 1991a), and any combination of these. Although imple menting cognitive flexibility theory i's not just a matter of, as Spiro et al. (1991a) state, using a computer to "connect everything with everything else" (p. 30), feedback can be designed into a hypertext system to lead the learner to approach concepts from new perspectives and to provide locator information when a learner feels lost in a "labyrinth of incidental or ad'hoc commections" (p. 30). Feedback traditionally has been used to allow the learner to evaluate preset goals through reinforcement of matching responses or through control of instruction. But in the constructivist view, evaluation provided by feedback would become more of a tool for self-analysis (Jonassen, 199 1 a).

Another constructivist inventio n is that of the microworld----~'a small but complete subset of reality in which one can go to learn about a specific domain through personal discovery and exploratioif' (cited in Rieber, 1992, p. 94) (see 12.3). Instructional applications of microworlds conform to Vygotsky's idea of the "zone of proximal development" (see 7A), in which learners who: are on the threshold of learning are often unable to attain understanding without some external intervention or assistance (Rieber, 1992).

Rieber contends that learning environments like microworlds should be designed with a "self-oriented feedback loop" (p. 100) that provides a rich and continual stream of information to help students establish and maintain goal setting and goal monitoring. Further, because many complex problems contain so many individual variables that can inundate a novice to the point of frustration, microworlds offer a way to structure the learning environment to a finite set of variables, something Piaget termed variable stepping (Rieber, 1992). Feedback received can be judged against a learner's individually defined goals. Rieber (1992) also suggests using a variety of feedback features to complement one another, such as presenting verbal feedback at the same time as visual feedback.

A report by Edwards (1991) focused on. how children used feedback from a computer microworld for transformational geometry to discover and correct instances of overgeneralizations that emerged as they solved problems with the microworld. Although there was a tendency towards symbolic overgeneralization in some activities, the children were able to use visual feedback from the microworld and discussions with their partners to correct their own errors.

A summary of the functions of feedback under a constructivist philosophy are presented in Figure 32-9. Researchers are encouraged to pursue the study of feedback under this paradigm.

The most recent synthesis of contemporary feedback models views feedback in the context of self-regulated learning (Butler & Winne, 1995). Butler and Winne (1995) propose a more elaborated examination of feedback that takes into account how feedback affects cognitive engagement with tasks and how engagement relates to achievement. Self-regulated students are aware of aspects of their own knowledge, beliefs, motivations, and cognitive processing, and the most effective learners are self-regulating. The model couples elements from traditional feedback research with processes involved in self-regulation. My view is that the Butler and Winne (1995) model quite possibly may supply the "missing link" between the findings presented in recent reviews (Bangert-Drowns et al., 1991; Kulhavy & Stock, 1989; Mory, 1992) and elements of motivation theory and constructivistic philosophies. They (Butler & Winne, 1995) point out that many studies of self4egulated learning (SRL) have looked at global or aggregate results of multiple SRL activities, rather than at individuali instances of self-regulation. They suggest a more "fine-grained analysis of feedback's roles in dynamic cognitive activities .that unfold during SRU (p. 247).

Figure 32-9. Suggested constructivist functions of feedback :Mory, 1995).

While most studies of feedback have focused on externally provided information, these researchers (1995) postulate that internal feedback is also inherent as selfregulated learners monitor their own engagement in tasks. The most effective learners develop their own distinct cognitive routines for creating this internal feedback, which in turn affects how the learrier will use information presented within feedback externally. Thus, the feedback serves a multidimensional role in aiding knowledge coostruction that fits into a model of self-regulation.

While not usually found in feedback or self4egulated learning (SRL) research, Butler and Winne (1995) cite several different areas of research and integrate these areas to aid in understanding the process of self-regulation as it relates to feedback. These include (1) how affectrelates to persistence during self-regulation, (2) what the role of leamer-generated feedback plays in decision making, (3) how students' beliefs affect learning, and (4) what beliefs learners have , in the process of conceptual change or restructuring when faced with misconceptions.

Self-regulation is the recursive process of interpreting information based on beliefs and knowledge, goal setting, and strategy applications to generate both mental and behavioral products (see Fig. 32-10). Mental products can include both cognitive and affective domains. Learners

monitor their own process of engagement and updated products through internal feedback. They then r ' einterpret the task and their own engagement, which then affects subsequent engagement. Modifications can include altering goals or setting new ones, reviewing and dapting their strategies of learning, and developing new skills. At this point, if external feedback is provided, additional information can be added to help the learner in this process (see Fig. 32-10).

32.5.7.1. Self-Regulated Engagement Four lines of research are featured in Butler and Winne's (1995) review of self-regulation. One is a model of self-regullation in terms of engagement and affect. Several researchers (Bandura, 1993; Carver & Scheier, 1990; Kuhl & Goschke, 1994; Mithaug, 1993; Zinunerman, 1989) have found that "students' goals couple with motivational beliefs and affective reactions to shape self-regulation" (Butler & Winne, 1995, p. 249). Positive affect results when progress is achieved faster than predicted, and negative affect results when the learner's rate of progress is slower than predicted. According to this model of SRL (Carver & Scheier, 1990), it is predicted that when learners make progress exactly as planned, the affect level that results is neutral rather than positive, and that under some conditions, achievement actually results in a negative affect. These affect levels influence future engagement on the task through the shaping of confidence judgments during the learner's internal monitoring process (Carver & Scheier, 1990; Eisenberger, 1992; Kuhl & Goschke, 1994).

Figure 32-10. A model of self-regulated leaming (from Butler & Winne, 1995). (From "Feedback and Self-regulated Learning," by D. L. Butler & P. H. )Vinne, 1995, Review of Educational Research 65, p. 248.) Copyright 1995 by the American Educational Research Association. Reprinted with permission.

32.5.7.2. A Lens Model. A second line of SRL research is from the viewpoint of what is terined a "Jens model," in which both task characteristics and students' progress on tasks are used to predict final performance. Traditional feedback studies focus on outcome feedback many times referred to as knowledge-of-results. While several studies do focus on adding elaborations to outcome information, most have ignored the role of giving learners guidance that can aid in their own self-regulation. Butler and Winne (1995) propose that data on students' perceptions of cues and their value, along with expectations for success and perceptions of actual achievement, can help researchers in knowing what to provide in elaborated feedback to support self-regulated engagement and to enhance self-calibration. Such feedback has been termed cognitive feedback (Balzer, Doherty & O'Connor, ~1989) and can provide learners information that links cues and achievement. Cognitive feedback includes (1) task validity feedback, (2) cognitive validity feedback, and (3) functional validity feedback. Task validity feedback includes information provided from an external source that describes that source. s perceived relationship between a task's cues and achievement (Butler & Winne, 1995; Elawar & Coino, 1985; Winne, 1989, 1992; Zellermayer, Salomon, Globerson & Givon, 1991). Cognitive validity feedback includes information describing the learner's own perceptions about the cue and achievement relationship (Butler & VA, nne, 1.995). And functional validity feedback describes the relationship between the leamer's own achievernen~ t estimation and actual end perforniance. In a review by Balzer and associates (1989), feedback that provided various forms of validityrelated information was found to be more effective than outcome feedback, and task validity feedback was somewhat more effective in supporting learning and problem solving than cognitive validity feedback information alone.

Several implications of examining feedback from a lens model viewpoint become evident. When providing outcome feedback, researchers should realize that the effectiveness of the feedback depends on several learner characteristics and behaviors. Students must be attentive to many cues, have accurate memories of cue features when receiving outcome feedback, and be strategic enough to generate effective internal feedback to themselves. Outcome feedback provides little guidance to the learner on how to self-regulate. However, when applying cognitive feedback, researchers should use information that helps students identify cues and monitor their own task engagement. This monitoring is an essential part of self-regulation.

32.5.7.3. Learners' Beliefs. A third line of SRL research examines the relationships between what the learner believes about learning, his or her use of strategies, and resulting performance (Schommer, 1990, 1993; Schommer, Crouse & Rhodes, 1992). Beliefs about learning can affect a student's persistent effort on a given task and goal orientation (Boekaerts, 1994; Carver & Scheier, 1990). These beliefs thus influence subsequent engagement on a task.

32.5.7.4. Misperceptions in Content. A learner's prior misconceptions about content area can hinder his or her subsequent revisions of that incorrect knowledge (Chinn & Brewer, 1993; Perkins & Simmons, 1988). While students can be receptive and correct misunderstandings through feedback, Chinn and Brewer (1993) identify six negative responses to feedback under such conditions. Students can (1) ignore the feedback, (2) reject feedback, (3) judge feedback to be irrelevant, (4) consider the feedback to be unrelated to the belief, (5) reinterpret the feedback to fit the misconceived belief, or (6) make superficial as opposed to fundamental changes in the erroneous belief. In this way, feedback is "filtered" through a learner's existing beliefs about the content.

Butler and Winne (1995) conclude that SRL is inherent in students' construction of knowledge. They assert that differentiating functions of feedback using a broadly framed model of self-regulation synthesizes the diversity of students on feedback and instruction. They identify the potential roles of feedback in remedying both strategy implementation failure and ineffective monitoring.

Students' knowledge and beliefs are linked with their self-regulated engagement in tasks. In addition to their epistemological beliefs, research on self-regulation also points to four other types of knowledge that learners bring to a task. These include domain knowledge, task knowledge, strategy knowledge,. and motivational beliefs. In terms of domain knowledge, students' strong incorrect knowledge structures within a domain result in erratic application of productive learning strategies (Burbules & Linn, 1988). As domain knowledge increases, students tend to acquire, use, and transfer cognitive strategies that support SRL (Salomon & Perkins, 1989). Task knowledge influences self-regulation as well, and learners' beliefs or interpretations of tasks can influence the goals they establish, as well as the cues attended to and acted on as they work on a task (Schommer, 1990).

Strategy knowledge results as students complete tasks. Winne and Butler (1994) identify three types of strategy knowledge. The first, declarative knowledge, involves stating what the strategy is. The second, procedural knowledge, involves how to use a particular strategy. And the third, conditional knowledge, addresses the utility of a strategy, such as when and where to use a strategy and how much effort will be required.

Finally, motivational knowledge involves Jearners' "beliefs about their capabilities to exercise control over their own level of functioning and over events that a#ect their lives" (Bandura, 1993, p. 118), referred to as se4~efflcacy. Self-efficacy affects the goals a learner will set, his or her commitment to those goals, decision making whi striving to reach those goals, and persistence (Bandura, 1993).

As mentioned in the research on motivation, students can adopt two types of task-related goals: le ing goals versus performance goals. Butler and Winne (1995) hypothesize that cognitive feedback containing inf Drmation about task cues will. be most effective when I iven to students that adopt learning goals. Further, the 4ffects of feedback depend on both the students' overall goo as well as the item-to-item change in their total knowledge as they review their wrong answers. The goals that stude# may be different from the goals intended by the i# designer, or researcher. When that is the case, feedback would probably have less stable or predictable Since goals are central in the process of selflearning, feedback must address the types of goals adopt and support their processes for prioritizatio tion, and maintenance of these goals (cited i & Winne, 1995).

In terms of students selecting and generating strategies to reach their goals, Winne (1982) notes four particular problems that students encounter. A learner may fa~l to recognize the conditions under which to employ the strategy. Secondly, learners may not understand the task or perceive the task goals and mismatch strategies to goals. Another problem occurs when students select good strategies but do not know how to apply them. And lastly, students may lack the motivation to expend effort in applying a strategy.

Monitoring is another important aspect of self-r~gulated learning. Monitoring generates internal feedback in the learner that links his or her past performance to the next successive task. The points of linkage are the prii'o:e times that feedback should be given to be most useful: (Butler & Winne, 1995).

The ideas put forth by Butler and Winne (1905) may well be the key to linking the two areas of motivation and constructivist philosophy as presented earlier 1 in this chapter. Through the blending of self-regulated learning research with research on feedback, both the motivational elements involved in learning and the philosophy of constructivism can be addressed. Their model (Butler & Winne, 1995) suggests that feedback is contexItualized according to the learner's prior knowledge and beliefs and, consequently, provides insufficient information to affect knowledge construction. They further suggest that for learning in authentic complex tasks, feedbacki should provide information about cognitive activities that promote learning and the relationships between cues and successive states of achievement.

Note also that the Kulhavy and Stock (1989) model emphasizing response certitude judgments adds credence to the notion that learners both set goals and monitor themselves. But Butler and Winne (1995) fine-tunes the issue by hypothesizing that students actually monitor their own calibration. Calibration is the extent to which monitoring creates accurate certitude judgments. They (Butler & Winne, 1995) suggest that high-confidence errors result in longer and more intense study of feedback because it is at this point that calibration is at its worst.

Traditional feedback research has been directed narrowly to the effects of feedback on achievement. The Butler and Winne (1995) model is a bridge allowing us to combine diverse studies on feedback, self-regulation, and instruction in such a way that future researchers have a schema for integrating instruction, self-regulation, feedback, and knowledge construction.

Perhaps one of the most important contributions to the use of adaptive feedback for facilitating learning lies in the advent of the microcomputer and its use for instruction. Unlike many of its technological predecessors, the computer has opened a door to interactivity, the precise recording of student response information, and the ability to adapt feedback and instruction to the changing needs of the learner within the interactive environment almost instantaneously. Further, recent developments in the use of multimedia and hypermedia open a vast set of questions for researchers to consider. For example, how does feedback function when presented via different modes of sensory input? Multimedia PCs common today involve the use of both auditory and visual stimuli to aid learning. What was once only possible through the integration of specialized media such as the interactive laserdisc now becomes more commonplace as newer technologies such as CD-ROM become increasingly commonplace and available. Hypertext and hypermedia designs await the learner using today's interactive CD software, with icons and "hotwords" linking vast amounts of information in the form of text, pictures, animations, and sounds.

A common problem with such open hypermedia environments is that learners often get lost along their exploratory way, unaware of how they were taken to the point at which they now rest. Navigation is just one of many variables to consider when examining such complex environments. Search (1994) suggests that if the communication potential of hypermedia is to reach its peak, designers must develop interfaces with orientation cues that help users navigate through large, multimedia databases. As she phrases it, "hypermedia computing is a temporal medium in which spatial relationships change dynamically, leaving the user with few references for orientation" (p. 369).

To understand adequately how the nature of computerbased learning has transformed, it is helpful to consider its evolution from its emergence in the 1960s as the programmed instruction movement to now. Jonassen (1993) notes that even early computer-assis lied instruction was merely programmed instruction delivered on a computer. The evolutionary path unfolded from programmed instruction, computer-based drills and tutorial~, adaptive tutorials , and simulations. An important conceptual framework for hypertext and hypermedia environments is presented by Jonassen (1993). The growth of hypertext, hypermedia, and multimedia since the 1980s has provided designers with the capabilities necessary to develop compl~x, content-oriented learning environments. In order to make such large quantities of information more accessible, a variety of conceptual models are being "mapped" onto these environments. As Jonassen (1993, p. 332) describes it:

Recent advances in learning theory have fueled a more rapid and extensive revolution in computer-supported learning systems. Rather than using' the computer as a delivery vehicle for displaying and purveying information, generative learning systems and kno ledge construction environments are designed to form partnerships with learners/users, to distribute the cognitive load and responsibility to the part of the learning systems that performs the best. Learners are engaged by these envirointellectual involvement in the learning process is essential.

They are no longer passive recepients of information ... they are actively involved in knowledge construction and meaning making. The computer's computational functionality is being used to support those processes rather than to present information.

The open architecture of hypermelia and multimedia have made them the platform of choice for implementing such knowledge construction environm4nts (see 24.6). The computers of the future will function as "intellectual toolkits for enhancing the intellectual and perceptual capacities of humans" (p. 333).

A useful framework for designing feedback by incorporating the powers of emerging instruction present, manipulate, control, and manag~ ities has been proposed by Hannafin, Ho (1993). They point out that emerging technologies provide the potential for a dramatic range of varied feedback not possible or practical to present before.

Feedback design helps in the ability to present information and support encoding. The range of esentation dimensions include visual, verbal, sensory, or multiple modalities. In order to optimize both individual pro~essing capabilities and technological potential requires anil expansion of our notion of both feedback and technology.

According to Hannafin et al. (1993), emerging technologies have provided six major areas of improvement for instruction: adaptability, realism, hypermedia, open-endedness, manipulability, and flexibility. To design feedback effectively requires the psychological, technological, and pedagogical foundations of lesson design (Hannafin, 1989).

Psychological foundations emphasize the role of the learner in processing inputs, organizing' and restructuring knowledge, and generating responses. Particularly relevant are processing requirements, the role of prior knowledge, the role of active processing, and strength encoding (Hannafin et al., 1993, p. 272).

Technological foundations concern the capabilities of the actual hardware and devices for providing output, receiving input, and processing data. Emphasis is on inputoutput capability, symbol manipulation, and management. In many instances, technological capabilities far exceed human processing capacity. Therefore, what is most important is not what the outer limits of technology are, but rather how to utilize those technological capacities (Hannafin et al., 1993).

Pedagogical foundations of design are rooted in beliefs about how to organize lesson knowledge, how to sequence activities in the lesson, and how to support the learner as he or she acquires knowledge. Many times, pedagogical factors are identified during a needs assessment or front-end analysis and include the resources and constraints of leamer, task, and setting characteristics (Hannafin et al., 1993).

As one might expect, even with emerging, high-profile technologies, distinctions of "good instruction, bad instruction" hold true (Hannafin et al., 1993). This includes the design of "good and bad' feedback within instruction as well.