23.5 Research and REALs

Current research issues within the field of REALs center on their overall effectiveness, methodological issues in conducting research, and making cognitive processes visible. Research conducted in the field parallels the development of REALs. REALs have developed in theory and implementation from content area teaching strategies used occasionally to integrated and comprehensive strategies guiding a whole curriculum. Research has evolved from comparing REALs with conventional teaching to studies focusing on how learners think and perform differently using REAL strategies. In the following literature survey, we begin with a look at the overall effectiveness of REALs and then move on to narrower issues related to learning and implementation.

23.5. 1 REAL vs. Conventional Instruction

Research comparing REALs with conventional instruction is generally quite favorable, finding REALs equal or superior to conventional instruction in teaching both problem-solving skills and content. I summarize several studies below that examine the overall effectiveness of a particular implementation of a REAL. Some are more successful than others. In my review, I attempt to bring out some of the methodological issues related to conducting research with REALs and suggest future research opportunities.

23.5.1.1. The Jasper Series. In one of the most extensive comparisons of a REAL versus conventional instruction, James Pellegrino and his CTGV colleagues (1991) tested the Jasper series in 16 schools in 9 states. Two teachers and a corporate support person had responsibility for implementing the program over the course of the 1990-1991 school year at each site. Teams received intensive 2-week training in the program before implementation. The CTGV group developed assessment strategies to evaluate Jasper's main goals for students: (1) to develop critical mathematical problem solving and reasoning skills, (2) to develop an appreciation of mathematics as a realistic part of their world and everyday problem solving, and (3) to develop various sets of specific mathematical knowledge and skills.

Pellegrino's group made a thorough attempt to measure changes in children's mathematical skills and attitudes over the course of a year. They used a battery of paper/pencil tests administered at the beginning and end of the school year to examine basic math concepts and content, attitudes toward math, word problem-solving skills, and higher-level problem-solving skills. Mini versions of these tests were administered at midyear. The Jasper classes were compared to children in classes that did not use the Jasper program.

23.5. 1. 1. 1. Math Attitudes. The CTGV team used a 35item questionnaire that covered 11 different categories

about attitudes and exploration and attribution behaviors. Although the reliabilities were low on the attitude scales (probably because of too few items), Jasper groups had more improved attitudes toward math at the end of the year compared to the control groups, believing that math was more fun and interesting than they did at the beginning of the year. However, despite the improved attitudes, Jasper students showed no greater desire to study or explore math than the control groups. Nor did the Jasper students attribute any more of their success to their own abilities than they did at the beginning of the year, for they still saw the teacher as the main force guiding their learning. It seems difficult to assess attributional and exploration behavior from a test. Future research might examine growth behaviors related in independence and personal responsibility to see if Jasper affects the development of intentional learning using observational techniques to supplement test instruments.

23.5.1.1.2. Basic Math Concepts. The Jasper content covered time and distance, area, perimeter, and volume, fraction/decimal. conversion, representation of fractions, and units of money, weight, and length. The Jasper students received no explicit instruction in these concepts, because the Jasper teams believed that Jasper provided a strong context in which students could anchor their learning of the concepts. Pellegrino used paper/pencil tests and word problems to examine students' skills. No main effects or interactions were found for gender. On tests of basic math abilities for decimals, fraction/decimals conversion, and area/ perimeter/volume, Jasper students had significantly larger increases in performance across the school year than the control groups. The study also used word problems to test "near transfer" of Jasper problem-solving skills. The word problems were a mix of one-step, two-step, and multistep problems. The Jasper group significantly outperformed the control group on all three problem types at year end, demonstrating greater problem-solving skills.

23.5.1.1.3. Planning Skills. One of the main goals of both Jasper and REALs is to teach students to become better problem solvers. Pellegrino et al., in the thoroughness that characterizes this study, created experimental planning problems to assess higher-level planning, subgoal comprehension, and calculation ability for the two Jasper subject areas: trip planning and sampling for business plans. The Jasper students were significantly more skilled at identifying the goal of the problem and in breaking the problem down into the smaller components or subgoals that lead to the solution.

23.5.1.1.4. Standardized Achievement Test Scores. Realizing that no innovation will be adopted in schools if it does not deal with the political realities of standardized testing, Pellegrino compared the standardized achievement scores of Jasper to the control group. This was a difficult task, and results were difficult to interpret, because not all of the sites had scores to compare, and five different tests were used across the sites. The Jasper students tended to perform slightly better than the control group, but not significantly so. Results are probably always going to be difficult to interpret because of the differences between the purposes of testing and the purposes of the instructional strategies used in REALs. However, the issue of standardized testing will not go away, and more structured studies need to be conducted.

23.5.1.1.5. Strengths and Weaknesses. Although experimental researchers may consider this study as "too uncontrolled" to be generalizable, I hold the contrary belief. This study is one of the most thorough and objectively reported that we have read in this field. It is a broad study using many schools and teachers incorporating qualitative and quantitative measures. Rather than trying to control every single factor within the classroom, it treated those factors as natural elements of the study. (I will examine later a study that tried to control too many factors.) When the researchers found anomalies in the quantitative results, they used their observations of the classrooms to find explanations. One of their most significant strengths is their effort to find out why their students scored as they did, not just what their students scored, although the researchers themselves criticize their measurement instruments as lacking sensitivity for this kind of research. They also need to rely less on paper-and-pencil tests and test the students in the way that Jasper works, in a team approach to problem solving. They also did not measure the effect of group participation or the growth individuals showed in their ability to work within groups. Finally, the reactions of administrators, colleagues, and parents to the program need examination.

23.5.1.2. Stoiber's Research in Teacher Education. Most research with REALs begins from a desire to develop instructional strategies that help students become more thoughtful and cognitively flexible so that they can perform better in realistic problem-solving situations. Often, the standard we hold our students to is that of the expert job performer. It is the expert's behavior that we wish to teach to our students. For example, Stoiber (1991) states that expert teachers are more thoughtful and have more developed knowledge structures to support reasoning and problem solving when managing classrooms. They also have a highly developed sense of responsibility for student motivation and achievement in the learning environment. In a study at the university level with preservice teacher education students, Stoiber found that a REAL strategy was more effective in developing reflective teachers than conventional instruction. Stoiber looked at 67 students in a teacher education program who had no experience in classroom teaching or management. The students were divided into technical, reflective, and control groups. Instruction in classroom management concepts was conducted over ten 50-minute sessions that met weekly.

23.5.1.2.1. Technical Condition, The technical approach is based on an orientation that portrays learning as acquiring concepts, principles, and techniques. The instructor emphasized Prescribed principles using a review-lecture-student participation format. Modeling and role playing were also used.

23.5.1.2.2. Reflective Condition The reflective approach, On the other hand, stresses the construction of concepts and principles based on existing knowledge structures. In the reflective condition, students analyzed classroom cases that focused on cognitive functions corresponding to three stages of teaching: preteaching (planning), interactive teaching, and postteaching. In the preteaching phase, students used self-inquiry methods to activate prior knowledge. They constructed mental representations of classrooms, including situations, decisions, actions, and outcomes visualizing situations and asking themselves what they would do in certain circumstances. In the interactive phase, students would "think aloud' while solving classroom case situations to help them become more conscious of the steps or strategies undertaken during classroom management. The postteaching phase involved self-evaluation comparing goals, intents, and images of teaching to teaching outcomes.

23.5.1.2.3. Control Condition In the control condition, students were instructed in education practices not related to classroom management. Students received excerpts of readings on an unrelated topic and examined vignettes and wrote short responses.

23.5.1.2.4. Instrumentation. Stoiber was creative in her measurement of student behaviors. She examined pedagogical reasoning and problem-solving performances. To examine pedagogical reasoning, she individually administered a video-stimulated interview of participants.

This measure examined participants pedagogical reasoning by stimulating their thinking about classroom management. The participants viewed a videotape of four classroom situations depicting classroom management problems (e.g., children whispering during a test, children not paying attention) role-played by an elementary teacher and her students. Each classroom problem vignette consisted of a classroom management incident prior to the teacher intervention. At the end of each videotape segment, participants were asked what action they would take if they were the teacher. Then to assess their pedagogical abilities, they were asked: "Why did you decide on this particular action/response?" (Stoiber, 1991, p. 134).

Interviews were audiotaped and coded for pedagogical reasoning by two advanced graduate students. The students' reasons were rated on a three-point scale: (1) contains no or limited reasons, (2) contains adequate or specific reasons supporting viewpoint or decision, and (3) contains elaborate or ethical reasons. Interviews were also coded for expressions of teacher responsibility related to (a) student affect, (b) student cognition, and (c) learning environment.

She measured problem-solving performance by examining student ability to solve management problems assessed using video-stimulated interview. Students watched a teacher deal inappropriately with the problem and were asked, "What suggestions would you offer for improving outcome?" Interview responses were coded in terms of five problem-solving strategies: (a) problem identification (identification or clarification of the problem), (b) generating alternative plans and solutions, (c) reflecting on the consequences of the plan's actions, (d) self-awareness and metacognitive activity during the interview, and (e) evaluative skills for reflecting on and critiquing teaching. Finally, they completed a problem-solving inventory.

23.5.1.2.5. Results. In the pedagogical reasoning analysis, the students in the reflective condition showed more expert-like behavior than those in the technical or control conditions. Their reasons for supporting their suggested actions were rated higher than were technical or control groups. They reported significantly more concern about student affect/attitudes more often than technical condition and mentioned being responsible for student cognitive performance significantly more often than control condition. They took significantly more responsibility for a positive learning environment than either technical or control groups.

The reflective group was also more sophisticated in its, problem-solving skills. The reflective group offered significantly more suggestions for alternative ways to handle the videotape situation than either other condition. The reflective group reflected significantly more often on the consequences of their decisions than the control group and reported significantly more alternatives for evaluating teaching practices than both the technical or control groups. The reflective group also exhibited more metacognitive awareness for improving poor pedagogical practices than the technical group and reported more frequently perceptions of themselves as solving problems in a manner associated with success than did the technical and control groups.

-The results of this study provide evidence that preservice teachers are capable of constructing concepts and developing the cognitive abilities needed to make sense of challenging classroom situations" (Stoiber, 1991, p. 137). Like the previous Jasper study, Stoiber's study is an excellent example of research with REALs. It combines qualitative and quantitative observations that not only measure the effects of the reasoning and problem solving skills but also explains why one group is more expert-like than another. However, like the Jasper study, we still need to see if this kind of development transfers to actual performance on the job.

23.5.1.3. Problem-Based Learning. Wilson and Cole, in Chapter 22, provide an extensive review of problem-based learning (PBL). I briefly summarize a review of PBL research in medical education by Coltrane (1993) to examine its overall effectiveness as a learning environment. The research on PBL is less conclusive than the research reported in the last two studies. Coltrane (1993) found too little data to determine conclusively that PBL is superior to conventional instruction in preparing physicians, though she found PBL at least equal to conventional instruction. The question of whether PBL is equal to or better than conventional instruction is important. REALs are usually viewed as more expensive in terms of time, effort, and resources than conventional, didactic instruction. Should PBL or REALs be adopted if they are not better than conventional instruction? Coltrane reports that Berkson (1993), in a meta-analysis of 12 PBL studies, found no direct advantage of one medical curriculum over another (PBL vs. regular). On the other hand, Mennin et al. (1993) found that PBL students outperformed conventional students in the later years of their study when self-directed, less-structured, and more independent learning experiences were encountered in their residencies. This last finding is critical, because, like the two previously discussed studies, it finds that PBL students improve the problem-solving skills that are a major component of high-quality, diagnostic performance. We also point out that the PBL research is more concerned with content acquisition than intentional learning and specific cognitive processes.

The concern for content coverage is also a concern of teachers thinking about adopting REAL teaching strategies. Teachers fear that their students will sacrifice breadth of content if they focus on the opportunity for depth that REALs afford. There is also the concern that the new teaching strategies will not prepare students for standardized achievement tests. However, the data in this area are generally positive. Dolmans (1993) found that PBL learning activities covered an average of 64% of intended course content. However, the coverage actually increased when students generated learning issues in response to their own needs, because half of those issues were judged relevant to course content. In terms of covering prescribed objectives, Rangachari (1991) found that students brainstorming about PBL problems identified and exceeded all of the faculty objectives. Blumberg and colleagues (1990) found adequate consistency between student issues and objectives generated by faculty. Coulson and Osborne (cf. Coltrane, 1994) discovered that PBL student groups identified an average of 61 % of faculty objectives deemed essential.

So, although the research on problem-based learning does not conclusively find that REAL strategies are better than conventional teaching in all regards, it does lend strong support. PBLs are at least equal to conventional instruction and probably better as the need for problem-solving and independent learning skills grow. Content and learning objective coverage may not be as systematic; however, research indicates that it is more than adequate.

23.5.1.4. Reciprocal Teaching. Another research question for REALs is whether they are useful for populations with special needs. Palincsar and Klenk (1992) investigated the effect of their intentional learning environment, reciprocal teaching, with young, at-risk children. "Young children with learning disabilities typically encounter difficulty with academic tasks requiring intentional effort and effective use of metacognitive skills--qualities that competent readers and writers possess" (Palincsar & Klenk, 1992, p. 211). Special education teachers often deal with these problems with greater decontextualization, isolating basic skills and drilling and practicing without context. However, Palincsar and Klenk contend that such instruction contributes to limited notions of literacy and fails to teach elements of intentional learning. So, they developed the reciprocal teaching strategies to place reading within a more meaningful context and to teach the intentional' learning skills many learning-disabled children do not use.

Reciprocal teaching emphasizes the social nature of learning with a focus on students learning many of the executive control functions usually considered the exclusive province of the teacher. Reciprocal teaching helps students learn these skills through appropriate questioning and dialogues about reading materials (see earlier in this chapter for a more detailed description of reciprocal teaching). The primary academic goal of Palincsar's program is to improve reading and listening-comprehension skills. In reciprocal teaching, students and teachers take turns leading discussions about the content of a text that they are trying to understand. The strategies used encourage the self-regulation and self-monitoring behaviors that promote intentional learning. Baseline studies indicate that at-risk students score typically below the 40th percentile in achievement and about 30% correct on independent measures of text comprehension on entering a reciprocal teaching study (Brown & Palincsar, 1989; Palincsar, 1990; Palincsar & Brown, 1989). After 3 months of instruction using reciprocal teaching, 80% met the criteria for success (75% to 80% correct) on measures of comprehension, recall, ability to draw inferences, ability to state the gist of material read, and application of knowledge acquired from the text. The students, both primary and middle school, maintained those gains 6 months later.

In another study (Palincsar & Klenk, 1992), results show that children with learning disabilities benefit significantly "from strategy instruction occurring within classroom cultures that support collaborative discourse, the flexible application of comprehension strategies, and appropriate, meaningful opportunities for reading and writing" (p. 211). In this study, which we will examine in detail, Palincsar and Klenk used 6 teachers and 30 first-grade children who typically scored below the 35th percentile on a standardized test of listening and comprehension. The children worked with a set of texts covering related science concepts.

23.5.1.4. 1. Pretests. The study began with a pretest to measure comprehension and knowledge of the science principles. The comprehension test was administered by reading a passage aloud to a student and then asking questions. This testing procedure was used regularly throughout the study. At the time of the pretest, the experimental and control children attained 47% correct. In identifying the theme of the passage, 29.2% of the experimental students were successful compared to 27.2% of the control group. A classification and sorting task was used to assess the children's ability to identify and use the analogy underlying the various topics. The teachers presented the children with pictures and asked them to put the pictures that go together in one pile and the other pictures in another pile. At the pretest, 43% (37% control) of the sorting decisions by the experimental groups were based on physical characteristics of the objects, and only 13% (14% control) were based on thematic characteristics.

23-5.1.4.2. Methodology. After the pretest, one passage was read to the students each day for 20 days. The basis of the procedure was reciprocal dialogues (see the actual article for sample dialogues). The children and teacher took turns leading discussions in which they questioned one another about the content of the passage. The group summarized the content, generated predictions about upcoming text, and clarified ambiguous information.

23.5.1.4.3. Results. The children in the experimental group made outstanding progress in 20 days. After the first 10 days of instruction, the experimental group attained 49.9% (37.7% control) correct on the comprehension measures. During the second 10 days, they attained 70.6% (39.5% control) correct. In their ability to identify the theme of the passage, the experimental group was correct 45.5% (14.9% control) during the first 10 days. During the second 10 days, this rose to 63.9% correct (10.5% control). On the classification task, the experimental group attained 53. 1 % correct (27% control) during the first half and 76.6% correct (17.3% control) during the second half

Palincsar's and Klenk's work is another model study - difficult and complicated, but valid and authentic. They designed the study carefully, trained the teachers, and conducted measurements that indicated thought and development in the students, not just knowledge acquisition. The measurements and tasks were authentic and fit within the normal range of classroom activities. Their study was thorough and included the most important dimensions of reciprocal teaching: discourse patterns among students and teachers, playfulness, the role of the teacher, and the role of the text. Palincsar and Klenk managed to conduct research with REALs in the most unobtrusive and authentic way possible.

23.5.1.5. Meaning Versus Algorithmic Math Teaching. In a year-long study of 40 eighth-grade mathematics classrooms comparing teaching with meaning, teaching with algorithmic strategies, and conventional teaching strategies, Sigurdson and Olson (1992) found significant effects for teaching with meaning. Both the algorithmic teaching and meaning strategies were considered innovative treatments. They defined algorithmic teaching as teaching math emphasizing computational performance and automatic application of mathematical rules. They defined "teaching with meaning" as "teaching in context." Students used physical and pictorial objects to represent mathematical concepts, placed concepts in familiar applications, and performed mathematical interpretations. The conventional classrooms were control groups using typical strategies. Included in their study was a long-term training program to help teachers learn either the algorithmic or meaning strategies, though the training program ran concurrently with the classroom treatments. Reported results are based only on the teachers who were considered successful implementers of the strategies.

The results of the study are a bit difficult to interpret, because they make a distinction between class ability and student ability, the relevance of which is hard to understand. It is also difficult to find the results that were "significant" over those that "showed trends." However, generally, the students in the classes of teachers who successfully implemented the teaching-with-meaning strategies had the highest achievement at the end of the year on a posttest. The above-average classes showed much greater gains in performance under the meaning strategies, while the below-average performed equally poorly under all strategies. In terms of individual abilities, higher-level students performed well under any strategy, while the middle-level ability students did better with the meaning strategy. The lower-level students did poorly with all strategies. Sigurdson's and Olson's study is a complex study fraught with all of the confounding problems of doing research in this area: individual ability levels, different classes, different teaching styles, the difficulty in achieving consistency among treatment classes, teacher training, and teacher cooperation. They may have had poorer results with the lower-ability students because their definition of "meaning" and "context" was severely limited. The teaching-with-meaning activities, though aimed at a deeper understanding of the concepts, lacked the authenticity level found in the Jasper series. The training program may have also had a negative effect on class achievement because some teachers may have grasped and implemented the strategies before others. The significance of their study is in the questions raised related to individual differences and abilities.

The above findings show that REALs work in a general way. Jasper teaches math and problem-solving skills. Stoiber's teacher education program taught preservice teachers to be more reflective and creative in solving classroom management problems. Problem-based learning covers the content and teaches problem-solving skills. Reciprocal teaching places reading in context and teaches students to be more reflective and metacognitively aware while reading. Teaching math with meaning is more successful for some learners than an algorithmic approach. Now I examine some more specific issues related to the effectiveness of REALs, including problem solving and transfer, tutors and content expertise, attitude toward content, and effects on cognitive structure.

23.5.2.1. Problem Solving and Transfer. One of the main assertions of REAL developers is that REALs improve problem-solving skills and enhance the likelihood that learning transfers to new situations. To accomplish this goal, REAL strategies do three things: (1) They make relevant problem-solving skills explicit; (2) they use context and authenticity to avoid the problem of inert knowledge; and (3) they have students deal with complex problems in complex ways.

23.5.2. 1. 1. Making Skills Explicit. Susan Williams and her colleagues at Vanderbilt (1992) found that without prior instruction in "what-if' thinking, students often have difficulty in knowing which aspects of their previous knowledge should remain intact and be used in a new problem. In her study, Jasper and control students watched a Jasper adventure. The Jasper students received instruction in the Jasper program, and the control students worked in a traditional math curriculum. Following instruction, students received a what-if problem related to the Jasper program. (A what-if program changed some variables in the original Jasper program.) Students were asked to talk aloud while solving the problem. The protocols were analyzed for the number of subgoals (16 necessary) attempted and the type of reasoning used. The Jasper students attempted to solve more subgoals than the control students. The Jasper work had made the problem-solving process more visible to them.

23.5.2.1.2. Inert Knowledge. The Jasper students in Williams's study also tried to make use of the declarative knowledge from the previous computations by applying it to the new problems. They may not have been right in their application, but the knowledge they learned provided a context for them to work in and did not remain inert. It helped them become active problem solvers. A number of other studies (Asch, 1969; Bransford et al., 1990; Brown et al., 1989; Gick & Holyoak, 1983; Ross, 1984; Stein, Way, Benningfield & Hedgecough, 1986) have found that when subjects are not explicitly informed about the relevance of knowledge to a problem, they do not use the knowledge-it remains inert. Bransford (1990) found that students who received statements that explicitly related acquired knowledge to problem statements were more likely to use that information during problem solving than subjects who did not receive statements drawing explicit links to the problem and their knowledge. Explicitly stating these relationships in the problem presentation helped students understand how information permitted a solution to that problem. Prior information that was recalled was then used more spontaneously.

23.5.2.1.3. Complexity. Real-world problems, that is, problems outside the classroom, are usually complex, requiring multiple substeps to solve. Yet, in classrooms, we have a tendency to limit problems to one or two steps. Students do not learn how to deal with multistep problems. The best way to teach complex problem solving, then, is to have students deal with complex problems in authentic contexts (CTGV, 1993d).

23.5.2.1.4. Far and Near Transfer The Cognition and Technology Group at Vanderbilt (1993d) found that making processes explicit, drawing explicit relationships with existing knowledge, and using natural complexity improved problem-solving skills facilitates transfer. In one study, the CTGV examined a high-achieving fifth-grade math class for near transfer. Students were assigned to a Jasper instruction group or a word problem group. They were given tests to assess learning and transfer before and after four I-hour instructional sessions. Students in the Jasper group scored much better on a mastery test of the Jasper program. The transfer test assessed transfer from Jasper to a highly similar problem. Students watched the transfer video and then solved the problem while talking aloud during an interview procedure. Their interviews were scored on whether they mentioned, attempted, or solved the major subproblems of the problem. The Jasper students scored higher than the word problem students. More than 75% of the Jasper-instructed group solved at least one of the top-level goals, compared with less than 20% of the word problem groups. These findings are also supported by Van Haneghan et al. (1992), Goldman with CTGV (1991), and Goldman et al. (199 1).

In a second study by the CTGV, the groups were tested on another transfer problem that was not isomorphically the same as the instructional problem-a far-transfer problem. Again the Jasper students scored significantly higher than the word-problem students. Jasper students mentioned, attempted, and solved a greater number of subproblems, even though the problem-solving steps were not the same as those encountered in the instructional program. This suggests that the Jasper-instructed students may have learned a general heuristic from the instruction.

On the whole, these findings suggest optimism about the value of REALs in teaching general problem-solving skills and transfer ability. Yet, there is still much work to do related to transfer. The distinction between near and far transfer is a continuum and not a dichotomy. The problem type, learner experience and intelligence, and problem context all play roles in determining what is near or far. What is far for one student may be near for another. What about transfer outside the classroom and across the curriculum? How successful are students in completely new environments? Are Jasper students able to apply anything they know to social studies, language arts, or science? The question of transfer is really the bottom line of REAL research. We justify REALs because our students are currently poor at transferring what they know. Are we improving?

23.5.2.2. 11itors and Content Expertise. REALs change teachers' roles by making teachers a greater part of the process of learning rather than the delivery of information. Teachers are often seen as equal participants in the learning process with students. This raises questions regarding their content expertise. Do teachers who are learning along with the students need to be experts in the content area? Or, since teachers are learning along with the students, do they only need to be experts in the learning process? The problem-based learning literature sheds some light on these questions.

Coltrane's (1993) review of PBL studies supports the contention that tutors should be experts in content. Students guided by content-expert tutors achieved better learning outcomes and spent more time on self-directed learning when their teachers were content experts. Expert tutors generated twice as many learning issues that were 3 times more congruent with case objectives than nonexpert tutors. However, Coltrane also found a disadvantage to expert tutors. Expert tutors tended to retain too much control of the learning process: They are more directive, speak more frequently and for longer periods, and more directly answered student questions, though they do suggest more discussion topics than nonexperts. Silver (1991) found that groups with expert tutors are less likely to be student directed.

It appears that good training of teachers may remedy the difficulties. Content knowledge balanced by an ability to "let go" and surrender control may be the best choice.

23.5.23. Attitude Toward Content Area. The Pellegrino et al. (1991) study (described above) found that students who used the REAL Jasper series had improved attitudes toward mathematics. In another, more loosely structured program, Myers (1992) studied the effects of using The Voyage of the Mimi in middle-school classes on female students. Mimi is a program that uses thirteen 15-n-linute video segments about a scientific expedition to study the Humpback whale. One of Mimi's main goals is to help integrate science across math, language arts, science, and social studies curricula boundaries promoting higher-level thinking activities. Students using the Mimi adventure spent the first period of the day viewing the day's adventure and then left for their other classes, rotating in and out of math, science, English, and social studies classrooms. The math teacher used a Mimi module on maps and navigation to study map making, measurement, distances, latitude and. longitude, triangulation, rations, and time-and-distance problems. The science classes studied the scientific method, food chains, and food webs. The English teacher had students keep journals, pretending to be a Mimi crew member. They also studied ethics of whaling, communication, and poetry. The social studies teacher conducted research on endangered species, paleontology, evolution, codes, and land forms.

The observer and teachers were struck by the enthusiasm students showed to studying Mimi-related subjects. Anchoring the study of the content areas into a realistic context was highly motivating. To study attitudes of female students toward the math and science content, Myers administered before-and-after attitude questionnaires. Myers administered the postexercise questionnaire 2 weeks following the unit. The prequestionnaire showed no difference in attitudes towards math and science between male and female students. The postquestionnaire showed no improvement from the start of the exercise till the end for the female students.

There are several possible reasons for this. First, the study lasted only 2 weeks, which is hardly enough time to effect lasting attitude change (see Chapter 34 for a discussion of media and attitude change). Secondly, in Myers' view, the teachers tried to maintain their traditional approach to instruction, adhering to behavioral objectives and county and state guidelines. They became more flexible through the unit, but were as much "guinea pigs" as the students. Third, he conducted no advance training for the teachers. If they were unprepared to change their attitudes, it is hardly reasonable to expect a great change among the students. Finally, I question the integrative nature of the study. The students had a common topic, but moved from class to class in a compartmentalized way. That is hardly an integrative approach to instruction, for in most integrative approaches, the teachers work together and in teams with their students. So, the effects of REALs on student attitudes toward content is still an open issue.

23.5.2.4. Cognitive Structure. The theoretical models of Ausubel (1968) and Gagne and White (1978) suggest that "connections" among propositions, images, episodes, and intellectual skills within a person's cognitive structure promote understanding and transfer. "These models also imply that, to understand new concepts, one must anchor the new concepts to existing structures" (Robertson, 1990, pp. 253-54). However, as David Jonassen (Jonassen, 1994a) is fond of saying, "The cerebra-scope has not yet been invented," so a clear picture of a person's cognitive structures that support conceptual understanding does not exist. However, research to visually approximate these cognitive structures has been conducted using pathfinder and multidimensional scaling analyses.

One of these methods uses cognitive maps representing the perceived relationships among several pairs of concepts associated with a specific topic using pathfinder techniques. Dunlap and Grabinger (1992) examined the effect of working within a computer-supported learning environment on participants' knowledge structures about learning environments. They conducted a class that studied the attributes of REALs. At the beginning of the class, they used KNOT-Mac (Interlink, 1990) to rate the degree of relatedness among 17 concepts about REALs. The KNOT-Mac program uses pathfinder analysis techniques to produce a map representing the user's knowledge structure. At the end of the class, the students rated the same 17 concepts. Average maps representing the class starting map, class ending map, and an experts' map were compared for similarity and complexity. Their findings indicate that after working within the learning environment the participants' ending map (die average map for the whole class) became much more complex and sophisticated than the average beginning map. However, the class map did not replicate the average experts' map. This was to be expected, since the experts had worked with the concepts in different ways and for a longer period of time than the students.

One of the unexpected benefits of using the maps was that the students became involved in analyzing their maps. First, they tended to ask, "Is my map right or wrong?" assuming the map was some kind of test. After being convinced that the map was no test, they began to ask themselves why certain concepts were linked and other weren't. They wanted to refine their understanding of the concepts.

Robertson (1990) implemented REAL strategies to teach physics problems related to Newton's second law. He used strategies to contextualize the students' learning to ensure that both internal and external associations were among the criteria attributes of a principle. Internal associations are connections among the criteria attributes of a principle. External associations refer to connections between the principle and everyday experiences or context. He used student-generated concept maps to help analyze the understandings that students had of Newton's second law and the related concepts. Robertson gave students practice with a number of problems. One set of problems was highly similar and another set was both similar and varied, requiring further transfer. The similar problems focused on internal connections, while the varied problems helped students make additional external connections. Using the maps, he rated the level of understanding students had of the system, looking at both internal and external connections. He found that 86.5% of variance for predicting success in solving the physics problems could be attributed to system-concept understanding developed through practice with several kinds of problems. The maps and problem performance indicated that students who practiced with both similar and varied problems to develop external and internal connections were more successful. The maps helped indicate and show visually the kinds of connections they made.

Pathfinder networks and concept maps can provide visual data to help interpret and analyze the progress of student thinking processes. They are complex and difficult to use and subject to student resistance because they take much time to generate. But, when used carefully and with student acceptance, they can be quite revealing and a valuable qualitative research tool.

On the whole, research into implementations of REALs shows positive effects for the REAL strategies. These positive effects show across ages, abilities, and content areas. However, research into the implementation of REALs is still young and developing. At the same time that teachers are trying to change their classrooms. from the teaching of decontextualized skills to the teaching of lifelong learning skills in context, educational research is struggling to move from a history of decontextualized experimental studies to more qualitative kinds of research within the natural context of the classroom. Both teachers and researchers have a lot to learn in terms of methodology in each area.