8.7 Indirect perception, mediated perception, and distributed cognition

Our species has invented various aids to perception, ways of improving, enhancing, or extending the pickup of information. The natural techniques of observation are supplemented by artificial techniques, using tools for perceiving by analogy with tools for performing Q. J. Gibson,1977/1982, p. 290; emphasis added).

If the great advantage that direct perception confers on organisms lies in an improved ability to detect affordances, a secondary benefit is that direct perception underlies "all less direct kinds of apprehension or cognition" (J. J. Gibson, 1977/1982, p. 289). Although he never developed a theory of indirect perception, Gibson clearly considered it an important topic, and be recognized degrees of directness and indirectness. His writing on this, issue, which consists mostly of unpublished notes, is inconsistent-as if he were still vacillating or cogitating about the idea.

Indirect perception is assisted perception: "the pickup of the invariant in stimulation after continued observation" (J. J. Gibson, 1979, p. 250). "The child who sees directly whether or not he can jump a ditch is aware of something more basic than is the child who has learned to say how wide it is in feet or meters" (J. J. Gibson, 1977/1982, p. 25 1).

Reed (1988, p. 315) points out that Gibson's preliminary efforts to distinguish direct and indirect forms of perception assumed that (a) ambient energy arrays within the environment (e.g., air pressure, light, gravity) provide the information that specifies affordance properties, and (b) the availability of these arrays has shaped the evolution of perceptual systems. Gibson thought that the exploratory actions of an organism engaged in perceiving energy arrays evidences the organism's "awareness" that stimulus information specifies affordance properties relevant to the requirements of the organism's niche.

On the other hand, Gibson recognized that instruments, pictures (see 26.2.3), and language can also be used to select, modify, and represent energy arrays.

Knowledge that has been put into words or, similarly, into numbers can be said to be explicit. It is rather different from the knowledge got by direct perception, by the simpler instruments, and by pictures. Not A information about the world can be put into words and numbers. Sometimes there are no words for what can be seen and captured in a picture. Is this because no verbal description is possible, or only because it has not yet been formulated? (J. J. Gibson, 1977/1982, p. 291).

Gibson (1977/1982) argued that symbols (i.e., notational symbols in Goodman's 1976 sense) are quite different from pictures and other visual arrays. Gibson believed that symbols constitute perhaps the most extreme form of indirect perception because:

... their meanings are attached by association. The meaning of an alphanumeric character or a combination of them fades away with prolonged visual fixation, unlike the meaning of a substance, surface, place, etc.... They make items that are unconnected with the rest of the world. Letters can stand for nonsense syllables (but there is no such thing as a nonsense place or a nonsense event) (p. 293).

Gibson, like other ecological psychologists, recognized the intellectual and constructive nature of indirect perception and, particularly, the important role that indirect perception plays in the creation and use of language.

Perceiving helps talking, and talking fixes the gains of perceiving. It is true that the adult who talks to a child can educate his attention to certain differences instead of others. It is true that when a child talks to himself be may enhance the tuning of his perception to certain differences rather than others. The range of possible discriminations is unlimited. Selection is inevitable. But this does not imply that the verbal fixing of information distorts the perception of the world. The ... observer can always observe more properties than he can describe Q. J. Gibson, 1966, p. 282).

We argued earlier that hum~in beings and other organisms benefit from thermodynamic leverage when they can off-load information storage and processing to nonbiological systems. Such off-loading requires improved perception--more reliable access to external information. It is not always easy, however, to estimate the costs associated with, respectively, internal representation and external representation, because the information is allocated dynamically. For example, after repeatedly forgetting some information item, one might decide to write it down (external, mediated representation) or, alternatively, to make a deliberate effort to memorize it (internal representation). Wise computer designers and users similarly attempt to optimize storage and processing of information between internal mechanisms (fast, but energy-consuming and volatile CPUs and RAMs) and external media (slow but energy efficient and stable CD-ROMs and backup tapes).

Where human beings are concerned, such dynamic allocation of storage and processing can be modeled as distributed cognitive tasks, defined by Zhang and Norman (1994) as "tasks that require the processing of information across the internal mind and the external environment" (p. 88). Zhang and Norman conceive of a distributed representation as a set of representations with (a) internal members, such as schemas, mental images, or propositions, and (b) external members such as physical symbols and external rules or constraints embedded in physical configurations. Representations are abstract structures with referents to the represented world.

Zhang and Norman (1994) propose a theoretical framework in which internal representations and external representations form a "distributed representational space" that represents the abstract structures and properties of the task in "abstract task space" (p. 90). They developed this framework to support rigorous and formal analysis of distributed cognitive tasks and to assist their investigations of "representational effects [in which] different isomorphic representations of a common formal structure can cause dramatically different cognitive behaviors" (p. 88). Figure 8-2 freely adapts elements of the Zhang-Norman framework (1994, Fig. 1, p. 90) by substituting MIROS for "internal representational space" and by further dividing external representational space into media (media space) and realia (real space).

We do not propose in this chapter to define rigorously mutually exclusive categories for media and realia. There are many types of hybrids. Museums, for example, often integrate realia with explanatory diagrams and audio. Recursion is also a problem: A portrait of George Washington is of interest as a physical artifact and also as a mediated representation of a real person; a spreadsheet program may include representations of itself in on-line multimedia tutorials. Our modification of the Zhang-Norman framework distinguishes real space from media space nevertheless, because there are often considerable differences between the affordance properties of realia and the affordance properties of media.

Figure 8-2. A framework for distributing cognition among media, realia, and mental-internal representations of situations (MIROS). Freely adapted from Zhang and Norman (1994, p. 90), this framework subdivides external representational space into media space (media) and real space (realia). The framework does not assume that corresponding elements in the three spaces will necessarily be isomorphic: in function or structure. On the contrary, there are usually profound differences.

Our adaptation of the Zhang-Norman model does not assume that corresponding elements in media space, real space, and internal representational space will necessarily be isomorphic in function or structure. On the contrary, there are often profound differences between the way corresponding information is structured in each space. Furthermore, as we argued earlier, MIROS vary in completeness and complexity. As Zhang and Norman (1994) demonstrated in their study of subjects attempting to solve the Tower of Hanoi problem, incongruent internal and external representations can interfere with task performance if critical aspects of the task structure, are dependent on such congruence.

Whatever the degree of correspondences between the structures of media, MIROS, and realia, external representations allow individuals to distribute some of the burden of storing and processing information to nonbiological systems--thus presumably improving thermodynamic efficiency. A key to intelligent interaction with a medium is to know how to optimize this distribution, to know when to manipulate a device, when to look something up (or write something down), and when to keep something in mind.

Of course media and realia can also support construction of MIROS that function more or less independently of interactions with external representational space. Salomon (1979, p. 234) used the term supplantation to refer to internalization of external representations as when the arithmetic operations of an abacus are internalized by expert abacus users. Salomon saw such learning by observation, not as a simple act of imitation or copying but as a process of elaboration involving recoding and mastery of constituent acts. ,

Distributed cognition points the way to the design of more efficient systems for supporting learning and performance. Yet the new representational systems offered by emergent computer and telecommunications technologies will challenge media researchers and designers to develop better models for determining which aspects of a given situation are best allocated to media or realia, and which are best allocated to MIROS.