It is relatively easy to find current literature extolling the virtues of multimedia or hypermedia environments. Among the commonly mentioned advantages are:*

• The ability to place learners in a context-rich enVironment
• An increase in learning due to the combination of text, graphics, full-motion video, and signs
• The ability to navigate complex nonlinear "hyperspace"
• An increase in motivation due to intrinsic aspects of the media

Desktop hardware and software have become more powerful, flexible, and sophisticated in the types of presentations that they can author and deliver. Moreover, such systems are within the budgets of many, if not most, K-12 classrooms. There has been a proliferation of authoring packages and CD-ROM-based programs that can deliver high-fidelity'sound, realistic color images in stills, graphics, and full-motion video. The central issue in this chapter, however, is whether multiple-channel presentations provided by multimedia environments contribute to an increase in the amount of learning.

The terms multiple-channel communication and cue summation are routinely used interchangeably in the literature. Is there a difference? The cue summation principle of learning theory predicts that learning is increased as the number of available cues or stimuli is increased (Severin, 1967a). Does this mean the addition of cues within a single-channel, such as adding color to a picture? Or does it mean adding cues across channels such as adding audio to a visual presentation? For the purposes of this review, cue summation will include both the addition of cues within and across channels. Therefore the multiple-channel communication research in this review may be subsumed under the cue summation theories. Supporting this approach is Miller's (1957) view concerning cue summation, which is frequently cited:

When cues from different modalities (or different cues within the same modality) are used simultaneously, they may either facilitate or interfere with each other. When cues elicit the same responses simultaneously, or different responses in the proper succession, they should summate to yield increased effectiveness. When the cues elicit incompatible responses, they should produce conflict and interference (p. 78).

Hoban (1949), in a summary of the instructional value of increasing the number of cues and/or realistic detail (which some call single-channel realism theory) in a visual presentation, concluded that the power of a medium of communication is determined by "the richness of the symbols employed" (p. 9) within that medium. These cues lead to greater understanding of the message by the audience.

Miller (1957) cites his views on the need to increase the number of cues in a presentation. He states that if one stimulus complex is to be identified versus another, the individual may use any number (even one) of available cues to make this discrimination. Increasing the number of available cues will increase the likelihood of an individual's making the correct discrimination over time and increasing the likelihood of a number of individuals making the correct discrimination simultaneously.

Dwyer (1978) suggests that the above views can be classified under the theoretical orientation collectively referred to as realism theories. The assumption is that:

Learning will be more complete as the number of cues in the learning situation increases. They suggest that an increase in realism in the existing cues in a learning situation increases the probability learning will be facilitated (p. 6).

(It should be noted that by making a learning situation more complex does not necessarily make it more realistic.)

Allen and Cooney (1963) suggest that age and maturity have effects on recall of information from multiple- or single-channel presentations. The mode of presentation has less effect on learning than does maturity Hsia (1969) studied the relationships between modalities and learner intelligence; he concluded that less-intelligent learners would be assisted positively if input, noise, and redundancy were controlled. Audiovisual (multiple-channel) presentations rather than single-channel presentations were suggested to optimize the information-processing rate of less-intelligent subjects. Further, Hsia recommended keeping cross-channel redundancy high in audiovisual (multiple-channel) presentations. Hsia (1968) similarly states that:

...in dual or multi-channel information-processing, dimensionality of information generally increases, and one channel provides cues and clues for the other, provided that the amount of information to be presented has not reached the capacity limit, thereby eliminating probable interference or information jamming. Increase in dimensionality usually results in the increase of information-processing (p. 326).

Severin (1967b) suggests that "multiple-channel communications appear to be superior to single-channel communications when relevant cues are summated across channels; neither is superior when redundant between channels, and are inferior when irrelevant cues are combined (presumably because irrelevant cues cause interference between them)" (p. 397).

Severin's theory of cue summation differs slightly from others in that he stresses the addition of "relevant' cues. This is somewhat of a caveat to the general theory of cue summation, which states that an increase in cues will summate in more learning. Severin 09670 also places emphasis on the use Of Pictorial presentations as the vehicle to add cues.

Van Mondfrans and Travers (1964) found that redundant information presented over two sense modalities (auditory Plus visual) resulted in no better learning than from either sense modality used alone. Severin (1967a) points out that the work of Van Mondfrans and Travers did not deal with nonredundant information presented over two channels. Their work looked at verbal material in both channelsOmitting the use of pictorial information.

Baggett and Ehrenfeucht (1983) reported that when college age Subjects are watching a film presentation and related information is presented simultaneously across two 'llediums-visual and auditory--4here is no competition for resources. When encoding visual and auditory information sequentially, the extraction of information is not increased. They concluded that synchronous visual/auditory input is an efficient way to present information. Baggett (1984) reported superiority of a simultaneous presentation of narrative and visuals over a presentation of the narration prior to corresponding visual sequence, but speech given slightly after a visual sequence resulted in recall just as good as a simultaneous presentation. Nugent (1982) studied content redundancy of content across three channels and found that when the content was the same, subjects learned equally as well from all modes, and by combining modes generally maximized learning.

It is not surprising that much of the multiple-channel (audiovisual) research has been conducted in the television venue, particularly with studies dealing with questions of redundancy (see 11.2.3). Findahl (1971), Reese (1983), and Drew and Grimes (1987) reported the superiority of redundant audio and video presentations in the recall and retention of verbal information and understanding of content. Likewise, Pezdek and Stevens (1984) found that with kindergarten students audio and video channels with "'matched' information was better for memory than when channels were 'mismatched."' They concluded that a high degree of redundancy helps learning in the audio channel and hinders the visual channel. With nonredundant material, the-students relied primarily on the video for meaning however. Calvert, Hudson, Watkins, and Wright (1982) reported that children learned more when verbal content was supported by understandable video than when abstract audio was accompanied by recognizable video.

Rolandelli (1989) reports that in television presentations, the visual mode is more important than the auditory mode when visual component competes with incongruent audio tract, but when visual superiority is confounded with complexity and comprehensibility, comprehensibility appears to be a more critical factor in viewer behavior. Audio can enhance comprehensibility by signaling what is worthy of attention and conveys information that can be understood independently of the visual mode (being present), In studies exploring irrelevant visual distractions (Festinger & Maccoby, 1964; Ostehouse & Brock, 1970; Bither, 1972), it was found that irrelevant visual distractions have an adverse effect on audio recall.

Lunisdaine and Gladstone (1958), Kale, Grosslight, and McIntyre (1955), and Kopstein and Roshal (1954) found the use of pictorial information or picture-word combinations more effective than words alone. Setting out to develop a hypothesis for these findings, Severin (1967a) suggested that the principles of cue summation and stimulus generalization accounted for improvement in learning. Stimulus generalization implies that "learning" improves as testing situations become more similar to -the presentation situation.

Additional studies have shown the superiority of the multiple-channel presentations of information. Severin (1967b, 1967c) reported that subjects receiving information with audio and related pictures received the highest scores of four treatments (sound only, picture only, sound and pictures, sound and unrelated pictures). He also reported finding that individual intelligence scores were less important in predicting learning than types of treatments. Hartman (1961a), in summarizing his study on multiple-channel effectiveness, indicated that "redundant information simultaneously presented by the audio and print channels is more effective in producing learning than the same information in either channel alone" (p. 42). Likewise, reviews of literature by Day and Beach (1950) that focused on the comparisons of audio and print channels, and the Hoban and Van Ormer studies (1950a) that concentrated on pictorial comparisons, concluded similar findings. However, Hartman (1961a) distinguished four relationships between multiplechannel messages and those on studies: redundant, related, unrelated, and contradictory. If multiple-channel messages are unrelated or contradictory, they compete with each other, and information interference is the result. That is why multiplechannel presentations were less effective in some studies. But if audio and visual messages were identical or closely related, they complement the other to form one thought and improve learning (Hanson, 1989; Ketcham & Heath, 1962). In educational practices, we seldom deliver unrelated or contradictory messages through multiple channels. Therefore, an improvement of leaming is expected by adopting the multiple-channel approach (Yang, 1993).

The implications of this work for development of multimedia products is considerable. It suggests that the addition of "bells and whistles" may contribute unrelated cues. As Severin (1967b) says: "If interference is accidentally introduced between channels, then much effort, time, and money is wasted, for one channel could then communicate more effectively" (p. 399). This work could provide advice for those engaged in the development of multimedia products for "atrisk" audiences. For these groups, less emphasis on print material, combined with the summation of cues using relevant material in the other channels, may be more appropriate. Smith and Smith (1966) critiqued earlier multiple-channel research (sometimes called audiovisual research). The Smiths stated:

Implicit in many of the older research designs which tried to make direct comparisons between different techniques was the assumptions that different types of instruction promoted the same type of learning-presumably the learning of verbal knowledge. These experimental comparisons usually were based on verbal criterion tests, for it was not realized that specialized audiovisual procedures might teach specialized nonverbal knowledge (p. 142).

Dwyer (1978) identified 19 factors that complicate interpretation and cause contradictory results of the single- and multiple-channel communication research studies. Some criticisms include weakness in experimental design, studies lacking hypotheses, research conducted in nonrealistic situations, and lack of relationship content used in one channel versus another.

Hartman (1961b), commenting on a review of 30 studies of channel comparisons, suggested that for presenting related information either through one or two channels, there is a strong indication of an advantage of combining channels. Severin (1967a) points out, however, that most of these studies were completed prior to 1940, and many contained poor research designs, lacked controls, and had test channel bias. Interference between channels, due to unrelated or opposing information, were not recognized in many of the studies. Severin (1967a) continues that a common practice among many communication researchers was to fill all channels in a multiple-channel situation with as much information (cues) as possible, with the expectation that this additional information would increase communication. The probability is quite high, however, that the additional information will only "evoke irreverent cues" (p. 234). Also see a strikingly similar statement by Hartman (1961b, p. 255).

Severin (1967a) attempts to explain the contradictory research findings of those who have studied multiple-channel and single-channel communication. Severin asks why some studies show an increase in learning in cross- (multiple) channel redundancy and others do not? Severin (1967a) suggests that educators sometimes use multiple channels without understanding the possibilities of interference between diem, and information may be presented via two channels and testing mode presented with only one channel. If, as Broadbent (1957) suggests, the central nervous system is a single system, separate presentations across two channels may not exceed its capacity, but together could overload and jam. Gulo and Baron (1965) and Williams and Ogilvie (1957) suggest that presentations do not always use the second channel to convey information and thus add nothing, not even redundancy, and might cause interference.

Hsia (1968) also questions the inconsistent findings. He feels that a major cause is the failure to take into account the capacity limit theorem and redundancy. First, redundancy causes information processing to fluctuate. Second, equivocation (loss of information) is caused by overloading the capacity limit. Hsia suggests that decreasing input information in accordance with the information-processing capacity will eliminate or reduce equivocation. Error, he submits, will be eliminated by adjusting redundancy to an optimum level so that maximunitransfer may take place.

Conway (1968), however, suggests that "the distinction between redundant and related information must now be regarded as an artifact of faulty conceptualization" (p. 409). He opines that equivalence in referential function is the criterion for redundancy. That is, "two items are redundant in that, as sign vehicles, they are interpreted to make reference in an equivalent fashion" (Conway, 1968, p. 409). T'wo important issues are implied in this discussion. First, Conway questions Severin's hypotheses concerning cue summation and stimulus generalization and the criteria upon which they are made. Second, Conway goes to some length in discussing whether relationships involving two signs or two modalities are redundant or related. If, as Conway proposes, most of the above relationships are redundant, as opposed to related, then there is no advantage in combining signs or sensory modalities. In refuting the hypothesis that presentations combining two sensory modalities are more efficient than either one of the modalities used alone, Conway cites findings from Van Mondfrans and Travers (1964), and from Severin (1967a, c). Severin's (1967a) position states there is no advantage in using "redundant" information over two modalities versus either one used alone. An example would be a presentation of the spoken word moose and the written word moose. Severin (1967a) hypothesizes that "related or relevant" presentations using two signs offer the greatest gain in communications. An example of the latter would be a picture of a moose and the written word moose, or a picture of a moose along with the spoken word moose.

Conway (1968), in an attempt to analyze the cue summation and stimulus generalization theories, tested word-pluspicture presentations against other conditions. He found that the present-picturehest-picture condition to be superior to those of present-word/test-word plus picture or those of present-word/test-word. He failed to find significance in the present-wordhest-picture and present-picture/test-word conditions. Conway suggests that the dual-coding theory (Paivio, 197 1) may account for the failure to support the stimulus generalization theory. For example:

... simple pictorial (line drawing) sign vehicles, although presented as single units, are, it is suggested, most likely to be coded and stored in two internal forms and therefore more likely than either word or word-plus-picture presentations to be readily assessed by the sign-vehicle presentations used to test memory (p. 412).

Using somewhat analogous reasoning to explain Van Mondfrans and Travers's (1964) failure to support an advantage to combined spoken- and printed-word presentations, Conway suggests that these messages are functionally equivalent and are already stored in word form. Therefore, using either spoken or printed-word presentations would be equal in learning to a combined presentation. It would follow that recall would also be equal under either stimulus, because the material is stored as a verbal string under both modes of presentation.

Much kinder to cue summation theory and Severin's (I 967a, 1967b) views is Hsia (1968, 197 1). He submits that "... tangible evidence suggests the possibility that when the amount of information to be processed is optimal, the audiovisual channels may be a more effective means of communication than either single charmer, (p. 246). Hsia (1971) makes a very thorough literature review of the discrete ranges of audio, visual, and audiovisual information processing rates and capacities. One of his conclusions is that combined audiovisual presentations produce more dimensionality than audio or visual alone. This dimensionality, he says, brings about an increase in information transfer within the information-processing capacity.

Hsia (1968) cautions, however, that multimodal information-processing seems to reach the overloading point faster than using single channels alone, especially when the between-channel redundancy is low. In essence, Hsia (1968) is proposing that designers remain cognizant of the principle that audiovisual cominunications will provide dimensionality and address individual-learner differences when used within the capacity of the nervous system. He also addresses individual-learner traits. For example, he cites research that supports use of the audio channel for young children, poor readers, and those of limited ability. In dealing with literate subjects, however, he provides evidence for using visual presentations. We could easily deduce that this information supports a need for multiple-channel presentations, especially when resources do not permit developing presentations for specific-learner types. Severin (1967a) makes the following predictions based on research when comparing single-channel communication and multiple-channel communication. Multiple-channel presentations that combine words and relevant visuals across channels will be the most effective and superior to single channels alone. This is due to cue summation across the channels. Multiple-channel communication with unrelated cues across both channels will cause interference, and thus single-channel presentations will be superior. Single-channel communication will be as effective as multiple-channel presentations when words (aurally and visually) are combined across channels.

Whether one subscribes to Severin's (1967) theory of using related multiple-channel communications, or the more generally held notion of using redundant information (Hsia, 1968), there is a considerable body of research supporting combined presentations (Levie & Lentz, 1982). From a review of over 155 experiments, Levie and Lentz (1982) suggest that: using attention-getting pictorials increase the possibility that material will be looked at; using text-redundant illustrations will facilitate learning the textual material; illustrations will help learners understand and remember readings; learners often need prompting to pay attention to critical information found in illustrations; learners' enjoyment and affective reactions may be evoked from illustrations; poor readers may benefit from illustrations; and learner-generated imaginal pictures are generally less useful than supplied illustrations.

Supporting both cue summation and stimulus generalization were two studies by Beck (1987). His findings indicated that labeled pictures used during instruction provided more effective encoding cues than arrowed or noncued pictures. During evaluation, the repetition of identical cues appeared to assist learners in retrieving critical information.

Rigney and Lutz (1976) found that the use of images significantly improved learning of complex concepts. Students also found the graphics versions to be more enjoyable. The enjoyment, it appears, increases involvement, so that students may acquire concepts from verbal instructional materials. Their research also supports Levie and Lentz's (1982) findings that supplied illustrations are better than user-generated imaginal pictures.

Mayer (1989) found evidence that the use of labeled illustrations helped students with limited prior knowledge of mechanical systems recall more explanative information and perform better on problem-solving transfer. He suggested that a meaningful learning model using illustrations helps focus attention on explanative textual information and to assimilate the information into useful mental models.

Mayer and Gallini (1990) tested two major features of illustrations that would assist learners in building mental models: system topology and component behavior. The former portrays each major system component; the latter portrays state changes in major components and the relationships of the components as the system functions. An example would be the major component of a braking system and the changes each component undergoes in relationship to the others as the system is employed. Findings supported their hypothesis that these illustrations would assist explanative recall and improve creative problem solving for low prior-knowledge learners.

Mayer and Anderson (1991) extended previous research (Mayer, 1989; Mayer & Gallini, 1990) by using voice narration and animation. While inconclusive, the results supported the theory that coordinated presentation of narrative and visuals (animations) (see 16.2.1) results in better performance on tests of creative problem solving than the word-before-pictures group. This research on integrated dual coding was adapted from Paivio's (1971) dual-code hypothesis. This extended theory posits that learners can build both visual and verbal representations as well as connections between them. Significant for designers was the finding that animation without narration had about the same effect as no instruction. Further, they found presenting unconnected words and pictures is not as useful as coordinated verbal narration simultaneous with animation.

Reynolds and Baker (1987) were interested in the notion of selective attention and its influence on using text and graphical representations. They found that texts with graphs, and texts without graphs, did not differ in degree of learning effect. Presenting materials on a computer, however, did increase attention and learning. Further, they found that interactive, graphical representation increased attention. The amount learned, although not significant, did show an increase. Their research suggested that when attention was increased, so was the amount of learning.

As noted earlier, questions over the superiority of individual channels have intrigued researchers for years. Conflicting results can be found which favor either channel. Katz and Deutsch (1963) and Travers (1964), for examples, reported results that supported the visual channel, over the auditory channel. However, Carterette and Jones (1967), Hartman (1961a, b), Henneman (1952), and Mowbray (1952) determined that auditory presentations were superior for young children and had more resistance to interference. Other researchers (Beagles-Roos & Gat, 1983; Meringoff, 1980) found that recall by children is comparable for visual and auditory modalities. However, Hayes, Kelly, and Mandel (1986) disagree and feel that verbal information recalled was incidental to the central plot of a televised program. Mudd and McConnick (1960) reported that, provided the information is related, auditory cues of various dimensions appreciably decrease the time involved in a visual-search task Warshaw (1978) reported on a series of experiments in which subjects were shown commercials with various juxtapositioning of different levels of audio and video information. He reported that when auditory information was presented without background video ( a blank screen), more content was recalled than when audio appeared simultaneously with relevant video, regardless of the level of information content in the second channel. Warshaw continued and stated that multiple-channel presentations do attract more attention than either channel alone, but perceptual interferences across multiple channels will hamper assimilation of the content.

Other studies supporting the single-channel, nervous system theory (Broadbent, 1958) ) found no difference between modalities (Baker & Alluisi, 1962; Hill & Hecher, 1966). Lorch, Bellock, and Augsback (1987) also noted that in televised presentations, children's recall of "central" content was comparable to audio only, visual only, or simultaneous across both modes. Grimes (1991) continues: In studies conducted with television where two channels-audio and visual-are highly redundant, people view the two channels as components of a single message. In a medium-redundancy situation, attention was shifted away from the visual channel and more attention was applied to the auditory channel. He reported contradictory results in a nonredundant presentation in one study in which the group attended to the video and in another study in which they did not. However, in the two experiments with nonredundant presentations, viewers' memory dropped for auditory messages and suggested low visual attention but high visual memory.