C H A P T E R
15
Visual Semiotics Theory
SANDR A MORIART Y
University of Colorado
In the movie 13 Days, a chronicle of the Cuban missile crisis, there is a scene in which Bob McNamara tries to explain to an admiral, who is in the process of escalating the military response, why his orders are wrong-headed. McNamara says that military options are a language and the Kennedy administration is trying to teach the Russians a new language.
He was referring to military actions as the communication of subtle diplomatic messages, rather than as a series of predictable military escalations. In other words, a bombing raid is a message with a number of meanings, just as the absence of a bombing raid is another type of message.
This riveting movie scene demonstrates how and why semiotics is an important tool to use in the process of excavating meanings in messages. Semiotics is the study of signs and codes, signs that are used in producing, conveying, and interpreting messages and the codes that govern their use. Jakobson used a broad definition when he says that semiotics is the communication of any messages whatever (1974, p. 32); however, Sebeok denned semiotics more tightly as the exchange of any messages whatever and the system of signs, or codes, which underlie them (1991, p. 60). Fiske added the idea of the generation of meaning to this definition (1990, p. 42). Messages, therefore, can be seen as made of signs and conveyed through sign systems called codes—in communication, meaning is derived only to the degree that the receiver of the message understands the code. In the scene quoted above, the admiral understands the military response code but McNamara understands the code of military response as a set of signs, or cues, used in the larger code system of Cold War negotiation.
Semiotics and semiology are two different but related approaches to a theory of signification—how these sign systems and codes work. Researchers in semiotics come from varied areas, such as communication, linguistics, anthropology, film study, literature, and marketing, as well as the natural sciences, where sign systems are studied in such areas as cellular biology and zoology. The focus of our concern is on the communication
227
aspects of a sign, and particularly the communication of nonverbal signs and the role of nonverbal signs in communication.
WHAT IS A SIGN
In semiotic theory, a sign is anything that stands for something else—that is, a sign stands for an object or concept (Hoopes, 1991, p. 141; Eco, 1986, p. 15). The Swiss linguist Ferdinand de Saussure, who is known as the father of European semiology, expressed this relationship in his Course in General Linguistics (1966) as the marriage between a sound or an image—called a signifier (Sr)—and the concept for which it stands (or content)—called the signified (Sd). The signifier is the form in which the content is expressed—the word, sound, picture, smell, or gesture. In explaining his theory of how the sign relationship works, Saussure used a tree and a drawing of a tree to both function as signifiers for the concept of "treeness," which he referred to as "arbor." (See Fig. 15.1.)
Saussure explained the closeness, or coherence, of these relationships with the metaphor of a sheet of paper as Fig. 15.2 illustrates. He said the paper itself was like the sign—one side of the sheet being the signified and the other side being the signifier.
Note that in his model both visual and verbal communication are equally represented as signifiers. As he develops his theories further, as we will see later in this discussion, his work becomes more highly logocentric and he uses language as a preferred model for sign systems, a viewpoint that privileges language at the expense of other sign systems, particularly visual perception and communication.
At the same time, around the beginning of the 20th century, as Saussure was developing his ideas of semiosis, American Charles S. Peirce was working on his model of knowledge and the way reality is represented in mind and thought. Peirce concluded in The Collected Papers that reality (and thoughts) can only be known through representation via signs, further that this signifying activity can best be explained through a three-part model of sign, interpretant, and object (1931, II, p. 135). The sign is equivalent to Saussure's signifier and the object is similar to Saussure's concept (see Fig. 15.3). The interpretant is the idea evoked in a person's mind by the sign. An association or personal experience, for example, contributes to my interpretation of the word tree—I see aspens in the fall turning gold.
Someone else might see a new spring green seedling being planted, a brilliant red/orange maple in the fall, or a stand of evergreens against a backdrop of snow. These personal
FIG. 15.1. Saussure's sign relationships.
15. VISUAL SEMIOTICS THEORY 229
FIG. 15.2. The relationships of signification.
FIG. 15.3. Peirce's model of a sign.
responses affect our interpretation of the sign and its object and lead to individualized interpretations.
Saussure's work was based in linguistics and his semiological contributions are impor
tant to critical analyses of how meaning operates in texts. Peirce's work, which became known as semiotics (although he never used that term), is even broader because of its epistomological focus. When Terence Hawkes in his analysis of the theory of signs de
clared that "Logic is only another name for semiotic, the formal doctrine of signs," (1977, p. 123) he reflected Peirce's larger concern for the important role signs play in the way we know things and think about things, rather than just talk about them. And because of Peirce's emphasis on representation as a key element in how a sign "stands for" its object, semiotics has become particularly useful to visual communication scholars who are, by definition, scholars and students of representation.
THE SIGN RELATIONSHIP
Important to both Saussure and Peirce is the notion of a relationship between the sign and object or signifier and signified. The question is, how does something come to stand for something else or how is the signifier connected to the signified? Different scholars have described this relationship in different ways.
Peirce, for example, used another triad—iconic, indexical, and symbolic—to ex
plain the nature of the relationships he identified for signs (1931, II, p. 157, IV, p. 359,
TABLE 15.1
Peirce's Three Types of Sign Relationships
Nature of Relationship Example
Iconic Resembles by mimesis— i.e., A photograph; a portrait
"looks like"
Indexical An indicator of the existence Smoke to fire; sympton of something to disease
Symbolic "Stands for" is understood A flag for a country; a mascot through convention for a team
II 143-144). As depicted in Table 15.1, an iconic sign is mimetic and resembles its object as a photograph does; it can be an indicator or material trace of its object (indexical) such as smoke indicates fire; or it can be connected to its object solely by convention as a flag symbolizes a country. Note how easy it is to analyze visuals using a Percian approach.
These three types of sign relationships lend themselves so easily to an explanation of how visual signs operate that Peirce's explanations of sign relationships are essentially all visual. One might conclude that for Peirce, visual communication is the underlying or master model for thought, rather than verbal language.
It is important to note that a sign can have various facets of these three types of meaning relationships. For example, when Sinead O'Connor tore up the photo of the Pope on a television show, the photo was iconic in that it looked like the person; it was indexical in that it indicated that the man actually exists; and, most importantly, the photo symbolized the institution of the Catholic Church with all its strictures and traditions. As Shaw pointed out, viewers responded with horror, but it wasn't because O'Connor destroyed an iconic image but rather because the photo—and the act of its destruction—carried such an overload of symbolic meaning (Shaw, 1994).
In addition to analyzing the type of sign relationship, semiotic scholars also consider several other important aspects of the sign relationship. At the heart of a theory of signs is the notion of oppositions and their role in creating meaning. The difference between motivated and unmotivated relationships and the difference between connotative and denotative meanings are also important points of analysis in semiotic theory.
Oppositions
At its most basic, the logic of a sign relationship, from a Saussurean or structuralist perspective, is based on a pattern of oppositions. As Muffoletto explained, meaning is derived in terms of "this is not that" (1994, p. 301). This is also the logic behind meaning, as well as formal definition with its structure of similarity and difference (Barthes, 1968, pp. 71-82); that is, to define something you state what category it belongs to, and then you delimit the definition by indicating what isn't included. The oppositional structures
231 15- VISUAL SEMIOTICS THEORY
contribute to meaning based on the same logic—you can understand "pretty" only by understanding "ugly." So a sign, particularly a visual sign, defines what something is, but also what it isn't. This relational structure is what makes it possible to communicate "rich"
(relative to "poor") with a simple picture of a status item such as a mansion. Meaning emerges through the play of difference between these oppositions.
Motivated and Unmotivated
Saussure, in his Course in General Linguistics, saw the relationship between signifier and signified as arbitrary and motivated—motivated meaning that there is no natural connec
tion between the two linguistics (Saussre, 1966, pp. 67-68). A word, for example (except perhaps for onomotopeia), has no connection with its concept. It is an arbitrary assign
ment that has to be learned. In visual communication, only symbolic visual signs would meet Saussure's criteria for signification—the way the mythical figure of the Jayhawk, for example, has come to stand for the University of Kansas.
Because he privileges language with its inherent arbitrariness as his model sign system, Saussure's logocentricity (Silverman, 1983, p. 5) limits the usefulness of his approach for visual communication scholars because so many visual signs are unmotivated, that is, natural and similar to their object. As Russian scholar Jakobson admitted in his critique of Saussure's approach (1985, p. 28), Peirce's schema is more flexible because it allows for unmotivated signification, such as that conveyed in iconic and indexical sign relationships.
Connotation and Denotation
Semiologist Roland Barthes (1968) and cultural studies theorist Stuart Hall (1999) have extended the concepts of signified (Sd) and signifier (Sr) to include connotation and denotation. Denotation is the direct, specific, or literal meaning we get from a sign. It is a description or representation of the signified—that is, language (or visual) specifically about the object. Connotation is meaning that is evoked by the object, that is, what it symbolizes on a subjective level. In Barthes' work, connotation reflects cultural meanings, mythologies, and ideologies. A connotative meaning is the "cultural baggage" attached to or associated with the object. It is derived from past experiences or repeated associations between a sign and its object. Barthes' theory is that there is a first and second level of meaning. Denotation is the starting point; meaning making then shifts to the second level where connotation takes over and delivers a richer experience of the meaning by engaging Peirce's interpretants.
Denotation and connotation are both used in visual communication. Clearly an iconic image, such as a portrait, is denotative. Lifestyle advertising, however, is an example of an arena where visual association is used to rapidly convey connotations. A product such as a car is first of all of the denotative level, a certain make of car with certain specific features.
On the second or connotative level, it is depicted as associated with a big house, elegant clothes, or an expensive meal to connote such things as quality good taste, premium price, status. Another product, a soft drink, is a particular type of beverage with a specific type of flavor. However, in the advertising it is associated with young teen boys having fun at a swimming hole or skateboard rink to connote such things as cool, not adult, but
more importantly, an escape from work and responsibility. In advertising, a huge part of message is conveyed through connotations that are delivered visually in subtle ways to extend what, on its face, appears to be a simple delivery of a simple message. Most of these connotative meanings operate in the nonverbal mode.
THE THEORY OF SIGNIFICATION
The objective of analyzing signification is to determine meaning—or a set of meanings.
The process of creating meaning is essentially that of locating or identifying the signified, that is, the concept, based on the cues given by the signifier, the sign. In a deterministic philosophy, the encoding is a process of "reading" sense data, a set of natural signs that represent "true conditions." In these empiricist traditions, language and other mediated signs can be literally true and the observer of the sign simply reacts to it (Anderson, 1993).
But the meaning-making process is more complex than the simple definition of a word or decipherment of a code, particularly for visual signs.
The concept of a process of perception adds points where individual factors, such as relevance, operate on the intended meaning. This subjectivity is further affected by social conditioning. Signification, then, is a complex process with various steps and levels, all of which offer points for individual personalization of meaning by the receiver. Signification, then, can be analyzed initially as a process of chains and shifts.
Chains
One charactertistic of the signification process is its ability to extend meaning either through the notion of the interpretant (Peirce) or the play of connotation (Barthes).
Typically what results from signification is a plurality of meanings. There may be one preferred reading—that is, that intended by the creator of the message, but there are often other meanings and levels of meaning that are created as the message is decoded by a recipient. That occurs for two reasons: either through the multiplication of signifiers or signifieds. Barthes referred to one type of plurality of signs as signifiers (Sr1,2,3) that
"float above" the signified (Sd) with the concept refusing to be anchored or constrained (Silverman, 1983, p. 32). In other words, there are a lot of ways to express an idea or depict something. Likewise, signifieds can also be interpreted in multiple ways. An example explained by Berger (1984, p. 15) are "Droodles," which are simple drawings (Sr) that can be read to mean many different things (Sd1,2,3).
The notion of plurality also suggests a chain of signification, the means by which these multiple signifieds and signifiers come to exist. Deeley describes meaning making as "a dynamic view of signification as a process" (1990, p. 23). Peirce explained that the meaning-making process is an infinite process of interpretation, what Eco called endless or "unlimited semiosis." Eco explained that because of similarities and resemblances
"everything can be connected with everything else, so that everything can be in turn either the expression or the content of any other thing." He also called this interpretive approach "Hermetic drift" (Deely, 1990, p. 24).
233 15- VISUAL SEMIOTICS THEORY
Variously called transactions or commutations by other scholars, this notion of chains is similar to how free association works. In this process a sign that functions as a signified then becomes a signifier that leads to other signifieds. This continues in an endless process of associational shifts. Derrida described the process of signification as leading from a signified that turns out to be another signifier in a process of endless chaining (Silverman, 1983).
Sr Sd Sr Sd Sr, etc.
Shifts
In addition to pluralities and chains, the process of signification also involves mental shift of several kinds. The first, as can be seen in the chain above, is a shift of focus from a signifier to its signified as the recipient of a message searches for meaning. However, that can also involve a shift of form, say from an image to a word to the underlying concept, idea, or thought. Meaning-making also involves a shift between the signified (Sd) and the interpretant as the interpreter searches for the meaning of a sign. According to Barthes, these shifts occur at a denotative level; however, as the interpretant process begins, then a more complex series of connotative shifts take over.
As mentioned earlier, in Barthes' original writings on connotation and denotation, most signs are thought to begin at the denotative level where the denotative signifier (DSr) and the denotative signified (DSd) combine as a sign unit to form the connotative signifier (CSr). Connotation, the second level, begins the process of generating additional meanings (CSd) and represents a paradigmatic shift. This is modeled in Fig. 15.4 and is similar to the way the interpretant functions in Peirce's model of a sign. For example, a photo of a house's elaborately carved wooden door may evoke not only an entry (the denotative meaning), but also wealth and art (CSd)—that is, people who can afford to own such an elaborate door probably live in fancy houses and appreciate "good" art.
II. CSign I. Dsign
DSr DSd
CSr CSd
FIG. 15.4. The relationship of denotation to connotation.
FIG. 15.5. The circular nature of denotation and connotation.
Perhaps it also symbolizes the door to such aspirations (CSr) for people who dream of such a lifestyle with all its attendant symbols and associations—fancy car, Rolex watch, designer clothes, servants, travel, and so on.
In later work, Barthes rejects the notion that denotation always conies first and, instead, sees denotation as both establishing and closing the reading (1974, p. 9). Another way to express the relationship between denotative and connotative is to begin with denotation—
by asking, what is it?—as Barthes did initially, then move to connotation—by asking, what does it suggest?—and finally wind up back at the beginning with confirmation of the denotation. In other words, using this adapted model of Barthes' work, the process of shifts, as depicted in Fig. 15.5, may be circular with denotation both beginning and ending the process.
Abduction
Another approach to the process of signification looks at the type of mental act involved in meaning-making as a complex effort of inference. This is based on the idea that logic, in its most basic definition, provides the principles that govern the validity of inference (Houser, 1991). In trying to better understand the role of the interpretant in meaning- making, scholars have explored the logic of interpretation as a complex inferential process based on Peirce's theory of abduction (Moriarty, 1996;Buchler, 1955,Peirce, 1931;Hoopes, 1991). In contrast to inductive (reasoning to) and deduction (reasoning from) logic, abduction is an inferential process that fashions conjectures based on "clues" that are available or conditions that are known. As Neiva said in his discussion of semiotics, rather than mimesis or convention, "the logic of signs is the logic of possibilities" (1999, p. 76). Anderson called this the "encyclopedic," rather than referential, character of signs.
He explained that "signs are the command to make meaning in a collectively recognizable, local performance of sense making" (1993, p. 212). Furthermore, he insisted that what something means is question of the present and appropriate to ask only at the site of considerable efforts and that must be answered anew at the next act of meaning construction (1993, p. 216).
235 15- VISUAL SEMIOTICS THEORY
In order to accumulate clues, the abductive process begins with observation, the bits and bytes of perception. It's similar to the way a doctor accumulates symptoms until he or she arrives at a diagnosis. Peirce described the formation of an abductive hypothesis as "act of insight/' the idea coming "like a flash"—the proverbial light bulb. In a more formal statement, abductive reasoning assembles the observations and attributes a variety of characteristics or conditions to a subject (the conjecture process) until a match is made and a conclusion can be stated. Another metaphor for abductive thinking is the semiotician as a detective, which Eco and Sebeok (1983) presented in their book, The Sign of Three. Beyond the linear forms of deductive and inductive logic, abductive reasoning more closely resembles massive parallel processing by computers, one that is not at all like language processing. The processing, however, is close to the nonlinear pattern used in perceptual processing, as Barry (1997) has explained in her book, Visual Intelligence.
Visual interpretation, however, involves more than simple inference. From semiotics we know that what is missing is sometimes as important as what is there. Eco (1979) suggested that a viewer goes through a process of "synthetic" inference that involves both denotative (realism, representation) and connotative (associations, attitudes, emotions) processes. Association unlocks this chaining process by deconstructing and unpacking the chains and shifts of signification. In this complex inferential process where information is being actively synthesized, an involved audience extends and fills in meaning, as well as decodes the meaning (Fry & Fry, 1983).
CODES: SYSTEMS OF SIGNS
A code is a set of rules (formula, ritual, genre) for usage or behavior, either stated or unstated. Eco suggested that it is not true that a code organizes signs but rather it is more correct to say that codes provide the rules that generate the signs (1979, p. 49). Language, for example, is governed by grammar and syntax. Other codes that have a more restricted language base include the Morse code and codes for flying and sailing. Driving has a set of rules of the road that a novice must master in order to get a driver's license. Every religion is a code for belief, as well as behavior. There is a code for dating that regulates acceptable and unacceptable behavior. Myths are considered by anthropologist Levi-Strauss (1967) to be a set of "coded" messages from a culture to its individuals. Parables operate the same way, particularly in a religious or ethical context. Myths and parables, in addition to manners and fashion, are examples of "cultural codes," which Silverman (1983, p. 36) defined as a conceptual system organized around key oppositions and equations that provide a basis for connotation. Approaching culture from the social perspective, Berger (1984, pp. 156-157) defines culture-codes as the secret structures that shape our behavior, or at least influence it. Culture itself is a code for human socializing.
Codes can be visual (road signs, sign language, Renaissance art), verbal (the alphabet, the Pledge of Allegiance), kinetic (cheer-leading), tactile (Braille), and auditory (musical notation, Morse code), as well as use other senses. The importance of a code is that it contributes meaning to a sign, as Muffoletto explained, "because of its historical uses in practice resulting in a seemingly fixed meaning, which has become part of a system of signs, or codes" (1994, p. 301). The code fixes the meaning of a sign.
A code can also be hidden or largely unnoticed, even by the people using the code.
That's how grammar operates as the background of language. Another example of a hidden code in a visual area is the film code that governs shots, camera direction, move
ment, and editing. Different kinds of shots (close-ups, reaction shots, etc.) have different meanings. Photographers and film and TV directors intuitively understand this "language of film" and use it deliberately to convey meanings separate from the words. A similar code governs the design principles used in graphic arts. The conventions of a genre—
soap opera, documentary, Western, science fiction—are also codes that define a type of communication. Semiotics scholars analyze these codes, as a way of "foregrounding the surreptitious signifying event" (Silverman, 1983, pp. 238-239), which brings the unnoticed code to the surface adding another layer of meaning to the interpretation of the passage.
Sorting Out the Language
This discussion of codes as sets of signs brings us to a problem in the discussion of semiotics, particularly visual semiotics, and that is the tangle of language-based termi
nology needed to express ideas and theories about how signification works. Signs are variously referred to as words, visuals, or images. And, although we recognize that a com
plex set of signs make up a sentence, photograph, painting, drawing, or movie scene, we lack the language to clearly articulate between and among these various types and levels of message carriers. Muffelotto, for example, described a picture as a collection of signs placed in relationship to each other (1994, p. 302). Film scholar Gretchen Barbatsis (1998) has been laboring with this problem and has recommended using the word "passsage"
(as in an earlier paragraph) for the complex constructions, be they verbal, visual, or moving images. A schema, then, for the language of semiotics, one that also separates the basic units of analysis, is as follows (see Table 15.2).
Types of Sign Systems
There are numerous ways to classify codes in addition to the elaborated/restricted distinc
tion. Berger, for example, identified five levels of codes in terms of their origin: universal, TABLE 15.2
The Language of Signification
Sign or Sign Unit Passage Code
A word (verbal cue) A sentence, paragraph Grammar / syntax A visual element (visual cue) An image, picture, or Layout, design principles
composition (drawing, photo)
A shot (visual cue) A scene (film, TV) Film /video conventions or
"grammar" of film
237 15. VISUAL SEMIOTICS THEORY
national, regional, local, and individual (1984, p. 158). Barthes' five levels of connotation—
semic (defines person or place), hermeneutic (sets up enigmas), proairetic (analyzes actions), symbolic (creates oppositional patterns), and cultural (identifies "truth")—
creates a kind of code for the structural analysis of connotation (see Silverman, 1983, p. 241). French semiologist Pierre Guiraud identified three kinds of sign systems: social, aesthetic, and logical (Barthes, 1982, p. 158).
In Saussure's approach to semiological theory, codes are analyzed in terms of mean
ing structures—paradigmatic or syntagmatic—using either synchronic or diachronic approaches or methods of analysis (Berger, 1982, pp. 23-32; Berger, 1984, p. 173;
Silverman, 1983, pp. 10-11,17). Although his approach is basically linguistic and only looks at how words and sentences are structured, this analytical schema has been broadened by other scholars to use with various types of meaning structures. A paradigmatic structure is the latent or hidden patterns that contribute to meaning usually focusing on similarity or difference (oppositions and associations) or subcodes (the conventions or "grammar"
of film). A syntagmatic structure is the chain of events, actions, or signs that lead to an understanding of how meaning is built up, such as words in a sentence or shots in a scene.
The meaning structure and its method of analysis can be paired. A synchronic analysis of the system of meaning looks at the similarities or oppositions in a paradigmatic structure.
A diachronic analysis is more historical in approach and looks at the way meaning unfolds over time in a syntagmatic structure through the sequencing of signs or events.
But Saussure also made the distinction between synchronic and diachronic meanings.
He explained that synchronic linguistics was concerned with the logical, psychologi
cal, and associational relationhips that bind together items and form a meaning system;
diachronic linguistics studies relationships that bind together successive terms in se
quence without forming a system (Saussure, 1966, pp. 99-100). It would seem, then, that paradigmatic meanings (latent, hidden) can be analyzed using a synchronic approach (logical, oppositional, and associational relationships), and sytagmatic meanings (chains and shifts) might best be analyzed using a diachronic approach (history, sequence). These relationships are depicted in Table 15.3.
A good example of visual research that analyzes both paradigmatic and syntagmatic structures using both synchonic and diachronic analysis is a study by Barbatsis of a
TABLE 15.3
The Structure of Meaning Relationships
Meaning Relationships Analytical Approach
Logic Paradigmatic Structures: latent, Synchronic Analysis: oppositions, hidden systems, and codes associations
History & Sequence Syntagmatic Structures: events, Diachronic Analysis: chains, shifts, actions, evolution successivity, how things unfold
over time
television commercial called The Harbor. Barbatsis looked at the structure of opposi
tions embedded in the design of the visual message, as well as the way they are se
quenced, to determine the propositions behind this essentially nonverbal commercial.
(See Chapter 19) The point is that an understanding of these types of structures and meth
ods of analyzing them provides another method for unpacking the levels of meaning in a sign or passage.
INTERPRETATION: CRACKING THE CODE
Semiotic meaning is complicated by the fact that, as Jakobson noted, "Language is a system of systems, an overall code which includes various subcodes" (1985, p. 30). That's true as well for visual messages, which are also composed of sets of signs and these sign sets are presented according to one or more sign systems, or codes. A house, for example, is a sign system. It has such elements as a kitchen, living room, bedrooms, bathroom, kitchen, garage, and basement. The kitchen, for example, encompasses various sets of codes that govern cooking, hygiene, food presentation, eating patterns, and storage.
Westerners, particularly Americans, understand this schema. However, a hundred years ago, an America would have a hard time understanding a garage and contemporary Americans may not have a clue about the purpose and function of a parlor. Asians may find some of these elements just as puzzling, and no one has clearly articulated the use of a kiva in the homes and villages of the ancestral puebloan peoples. One of Barthes' most respected books, The Empire of Signs (1982), documents Japanese culture from a semiotic viewpoint analyzing the meaning of practices, activities, and other forms that have almost unintelligble meanings for Westerners.
The goal of a semiotic analysis of sign systems is focused on interpreting the inter
play of a multiplicity of codes. In other words, the process of signification, or semiotic interpretation, involves the deconstruction of the various sign systems and layers of codes that are operating in a passage (picture, image, etc.). Like peeling an onion, one sign system is studied, then another, then another, until reaching the essence of the sign—a process of unlayering, or working backward from the sign units to the sign's position in the various systems, to analyze how meanings are built up by the mul
tiplication of signs and codes. This is the classic approach to art appreciation, which looks at, among other things, the figures and settings, the foregrounding and back- grounding of elements, the aesthetic cues, the perceptual cues (such as perspective), the lighting, the colors, the historical cues, as well as the mythical or allegorical associated meanings.
Structure
The notion of a code as a system of signs reflects a structuralist view of culture. Crack
ing the code, from a structuralist perspective, means deconstructing the pattern of the underlying rules that govern the system. An example from the verbal arena is the anal
ysis of speech as elaborated or restricted—elaborated being more complex and open to interpretation while restricted is more simple and predictable.
239 15. VISUA L SEMIOTICS THEORY
In an open system the meaning is less structured and codified. Visuals are more sus
ceptible to varied interpretations because of their open systems of meaning; more so than are words whose meaning is more likely to be fixed through dictionaries and con
ventional, agreed on uses. Because of the openness of visual communication structures, there are also more opportunities for chains and shifts. An example is an advertisement for a lifestyle product, such as an exotic vacation, which invites the viewers to project themselves and their lives into the ad situation in the visual and make it their own. In contrast, an ad for a cold medicine will typically use visual and verbal elements to spell out exactly when and how the product is to be used.
Likewise, deduction and induction are more closed because of the rigid structure of the underlying logic; abduction, in contrast, is more open to interpretation. Such personalized interpretation is invited in art, particularly abstract and modern art, where viewers are invited to create their own meanings. Renaissance painting, however, may be more closed because the composition can be more accurately interpreted if the viewer is trained in deconstructing the code.
ISSUE: THE BOUNDARIES OF VISUAL COMMUNICATION
This discussion has focused on signs and sign systems; however, there is another term that generates a critical issue in understanding visual semiotics, and that is the word signal. A signal, such as a stoplight, is a message communicated through signs that incite action. The process of signaling is used to relate or make something known. Consider a red blinking light at a train crossing where viewers of the signal know from previous experience that the red blinking light warns of an oncoming train. The question is: Is that communication and can it logically be approached through semiotic analysis? The reason this question arises is because the signifying process is based on recognition, rather than interpretation.
In analyzing the scope and breadth of semiotics Eco (1979) and Sebeok (1991) both identified different types of natural semiotic systems that function in a communicative way as signaling systems. For example, zoosemiosis includes animal communication—
all those puzzling questions we ask about how bats fly, how flocks of birds turn syn
chronously, the meaning of dolphin and whale sounds, and the mapping communicated by the honeybee's dance. In the more general area of biosemiosis, a biological theory of communication operates through the signaling process of cells. A particular kind of biocommunication called endosemiotics studies communication at the molecular level.
One might ask how cells can be described as communicating organisms. Molecular and cellular biologists, however, are involved in puzzling out signaling systems used both inside a cell as proteins and acids are moved around, and between cells as they create complex molecular and cellular structures. But is such signaling truly communication?
As mentioned earlier in the section on definitions of signs and semiotics, Jakobson uses a broad definition when he said that semiotics is the communication of any messages whatever. Using that approach, nature can be seen as a system of coded signs. Eco made the argument that the roots of semiotics lie far back in time with hunters and trackers who could read the signs of nature. More recently, Bill McKibbon in his book The Age of
Missing Information (1992) made the point that our ancestors were much more attuned to the land, to weather, to animal behavior, and to nature than we are now. They knew how to read the cues and clues (Pence's indexical signs) much better than we can because most of us are not familiar any more with these code systems.
Other scholars, however, make a distinction between human communication and other forms of what is essentially signaling based on recognition. University of Penn
sylvania scholars Sol Worth and Larry Gross (1981) analyzed communication in terms of what they call "interpretive strategies." In other words, signaling that does not call for interpretation is of less interest to them as a communicative behavior. They make a distinction between sign events, events that generate an interpretive response, and nonsign events, events that are latent, hidden, or coded transparently. Then they distinguish be
tween natural sign events (Peirce's indexical signs and Eco and Sebeok's zoosemiosis) and symbolic sign events, such as words. Natural sign events, such as a footprint, can be in
terpreted as a sign of something but it does not necessarily represent an intentional piece of communication. In other words, communication is grounded in intended meanings that need to be interpreted. Communication, then, requires an interpretive strategy on the part of both the sender and the receiver. A stoplight, then, is communication, but a footprint or signaling processes at the cellular level aren't.
Obviously this issue is important to visual communication scholars because it defines the boundaries of their investigations. Is intention a requirement for a communication to occur? In spite of Peirce's argument in support of the importance of indexical signs, are they to be ignored by visual communication scholars because some, if not most, are natural signs and lack intention? Or is the communication in the mind of the receiver—
that is, if a sign generates meaning, isn't that communication? The purpose of this chapter is more to raise such issues than to resolve them. Further theory-building will help set the parameters for better informed research in this area.
REFERENCES
Anderson,]. A. (1993). The role of interpretation in communication theory. In Nikos Metallinos (Ed.), Verbo
visual literacy: Understanding and applying new educational communication media technologies (pp. 211-222).
IVLA International Symposium, Delphi, Greece.
Barbatsis, G. (1998, June). Analyzing meaning inform: Domestic comedy's angle of refraction. Paper presented at the Visual Communication Conference, Winter Park, Colorado.
Barry, A. M. (1997). Visual intelligence. Albany: State University of New York Press.
Barthes, R. (1968). Elements ofsemiology. Annette Lavers & Colin Smith (Trans.). New York: Hill and Wang.
Barthes, R. (1974). S/Z. Richard Miller (Trans.). New York: Hill and Wang.
Barthes, R. (1982). The empire of signs. New York: Hill and Wang.
Berger, A. A. (1982). Media analysis techniques. Newbury Park, CA: Sage.
Berger, A. A. (1984). Signs in contemporary culture. New York: Longman.
Buchler, J. (Ed.). (1955). Philosophial writings of perice. New York: Dover.
Deeley, J. (1990). Basics of semiotics. Bloomington: Indiana University Press.
Eco, U. (1979). A theory of semiotics. Bloomington: Indiana University Press.
Eco, U. (1986). Semiotics and the philosophy oflanguage. Bloomington: Indiana University Press, Midland Book Edition.
Eco, U. (1990). The limits of interpretation. Bloomington: Indiana University Press.
Eco, U, & Sebeok, T. A. (Eds.). (1983). The sign of three. Bloomington: Indiana University Press.
241 15- VISUAL SEMIOTICS THEORY
Fiske, J. (1990). Introduction to communication (2nd ed.). London: Routledge.
Fry, D. L., & Fry, Y H. (1983). A semiotic model for the study of mass communication. Communication Yearbook, 9, 443-462.
Griffin, M. (1999). Camera as witness, image as sign: The study of visual communication. Communication Yearbook, 24, 433-463.
Hall, S. (1993). Encoding, decoding. In Simon During (Ed.), The cultural studies reader (pp. 90-103). New York and London: Routledge.
Hall, S. (1999). Introduction to Part III: Looking and subjectivity. In Jessica Evans & Stuart Hall (Eds.), Visual culture: the reader. London: Sage Publications.
Hawkes, T. (1977). Structuralism and semiotics. Berkeley: University of California Press.
Hoopes, J. (1991). Peirce on signs. Chapel Hill, NC: The University of North Carolina Press.
Houser, N. (1991). Competiting icons. In John Deely & Terry Prewitt (Eds.), Semiotics 1991 (pp. 20-26). New York: University Press of America.
Jakobson, R. (1974). Main trends in the science of language. New York: Harper & Row.
Jakobson, R. (1985). Verbal art, verbal sign, verbal time. Minneapolis: University of Minnesota Press.
Levi-Strauss, C. (1967). Structural anthropology. Garden City, NY: Doubleday.
McKibbon, B. (1993). The age of missing information. New York: Button.
Moriarty, S. (1996). Abduction: A theory of visual interpretation." Communication Theory, 6(2), 167-187.
Muffoletto, R. (1994). Representations: You, me, and them. In David (Mike) Moore & Francis Dwyer (Eds.), Visual literacy (pp. 295-312). Englewood Cliffs, NJ: Educational Technology Publications.
Neiva, E. (1999). Redefining the image: Mimesis, convention, and semiotics. Communication theory, 9(1) 75-91.
Peirce, C. S. (1931). Collected papers. Vols I-VIII. Charles Hartshorne & Paul Weiss (Eds.). Cambridge, MA:
Harvard University Press.
Saussure, F. de. (1966). Course in general linguistics. Wade Baskin (Trans.). New York: McGraw-Hill.
Sebeok, T. A. (1991).A sign is just a sign. Bloomington: Indiana University Press.
Shaw, D. (1994). Getting to the pulp of it: Sinead O'Connor and the iconoclastic pulpit. Paper presented at the Speech Communication Association Conference, New Orleans.
Silverman, K. (1983). The subject of semiotics. New York and Oxford: Oxford University Press.
Worth, S., & Gross L. (1981). Symbolic strategies. In Larry Gross (Ed.), Studying visual communication (pp. 134-147). Philadelphia: University of Pennsylvania Press.
