(2000)1462
Visual semiotics
The feasibility of such a domain as visual semiotics, a speciality purportedly concerned with the investigation of all kinds of meaning conveyed by means of the visual senses, may well be doubted: following one common interpretation, it should be excluded by the structuralist conception according to which form, not substance, is relevant to meaning. Yet, Saussure’s preoccupation with the linearity of speech, as against the multidimensionality of pictures, Jakobson’s interest in the differences between auditory and visual signs, and, more generally, the Prague school model, according to which the receiver of the work of art is called upon to transform a sign into a concrete perceptual object, certainly suggest a different conception. Indeed, Lessing already was essentially concerned with the location of visual signs in space, as opposed to the projection of auditory signs into time. According to the Greimas school, however, all meaning is of a kind, so not only do pictures and literature manifest the same organising principles, but, more broadly, visuality and aurality must be taken to be identical on a deeper level.
In recent semiotics, Kümmel (1969) has written a rather haphazard catalogue of visually conveyed signs. The first to propound a division of all signs according to the sense modalities which they address was Thomas Sebeok (1976). Preziosi (1983) then ranged architecture among the several devices of visual communication, opposing them in some respects to verbal language. According to Fernande Saint-Martin (1987), visual semiotics comprises the study of pictorial art, sculpture, and architecture. This means that she ignores all visual signs which are not, in our culture, considered to be artistic (in spite of some passing remarks on children’s drawings in other publications). In actual fact, pictures, sculpture and architecture also are the only domains, with an emphasis on the first one, which are covered by Groupe µ’s (1992) recent book the title of which announces that it is concerned with visual signs.
To the extent that pictorial semiotics has been well-advised to turn recently to perceptual psychology in search of its foundations, we must suppose there to be some general organising principles of pictorial and other visual signs which are relevant to their transmission of meaning. If so, however, it will be necessary to distinguish those domains which are intrinsically visual in organisation, from those in which meanings which are differently constituted are merely secondarily conveyed by visual means.
On the other hand, from the point of view of Hjelmslevean semiotics, we would normally not expect visuality, being a mere “substance” or even “matter”, to determine any relevant categorisations of semiotic means. In their dictionary, Greimas & Courtés actually claims that sense modalities, identified with the expression substance, are not pertinent for semiotics, and this is no doubt the reason for visuality being one of the many layers between the unique picture and signification per se being left out of consideration in Floch’s analyses.
This type of argument is based on a confusion of the terms “substance” and “matière”, as employed by Hjelmslev, and in their ordinary usage. In fact, the term “matière”, to Hjelmslev, is that which is unknowable, and, as a consequence, not susceptible of being analysed; that is, it is the residue of the analysis; and “substance”, which, in the earlier texts, is the term used for “matière” in the above-mentioned sense, stands, in the later works, for the combination of “matière” and “form”. Thus, “substance”, in the early works, and “matter” later, simply means “that which is not pertinent relative to the other plane of the sign” (see discussion in Sonesson 1989,II.4. and 1988); it does not necessarily stand for matter in the sense of ordinary language, that is, the material of which something is made, or the sense modality. If the material or the sense modality turns out to be relevant in relation to the other plane of signification, it becomes form (from Hjelmslev’s standpoint, this is what happens in connotational language).
of the vocal channel by which it was once exclusively conveyed. That is, the qualities of the visual sense modality are of interest to semiotics, to the extent that they specify formal properties embodied in each system addressed to that particular sense. Although Groupe µ seems unaware of it, Hjelmslev (1953) does not reason differently when he posits different “forms” for written and spoken language.
If, however, properties imposed by their mode of communication are only some among several traits defining these signs, as is the case of the linearity of verbal language, one may well wonder whether we are really justified in making visuality into a subdivision of semiotics. There may, moreover, be other, perhaps more fundamental division blocks of semiosis, of which pictures and some other visual signs form a part, such as, for instance, that of iconicity. On the other hand, it is conceivable that some more decisive argument could be advanced for privileging the domain of visuality over other possible divisions.
But to the extent that there is a legitimate domain of visual semiotics, it should comprehend much more the sacred trinity of art history, painting (to which drawing, photography, and so on, have been assimilated), sculpture, and architecture, of which Groupe µ, just like Saint-Martin (1987) seems to be the victim, in spite of the promise made in the introduction (p.12ff) to ignore received categories such as art. As soon as we leave the traditional divisions of art history behind, this analysis has a very limited value. Thus, sculpture should be compared semiotically similar objects like the tailor’s dummy, and the like. At one point (p.405f), the authors are suddenly reminded of marionettes, considered as a kind of sculpture to which movement has been added. But why not also add the ballet dancer, whose art is certainly visual?
The real issues of visual semiotics may then turn out to be still, or rather again, those characterised by Lessing (cf. Sonesson 1988): if language is better adapted to the rendering of temporal succession, while pictures lend themselves more readily to deployment in space, then how is that visuality and narrativity, as many critics of television and more recent media have suggested, concurrently invade our culture? The interrelations of visual and auditory signs (which are not simply verbal either) certainly deserves more serious consideration than has been given them by those propounding the domain of visual semiotics.
Göran Sonesson
Bibliography:
Kümmel, Peter, Struktur und Funktion sichtbarer Zeichen Quickborn. Schnelle 1969
Greimas, A.J. & Corutés, J., Sémiotique. Dictionnaire raiaonné de la théorie du langage. Paris: Hachette 1979.
Groupe µ, Traité du signe visuel. Paris: Seuil 1992.
Hjelmslev, Louis, Introduction à la discusssion générale des problèmes relatifs à la phonologie de langues mortes. In Acta Congressus Madvigiani: Proceedings of the Second International Congress of Classical Studies, Copenhagen 1954, Copenhagen 1954, 101-113.
Jakobson, Roman, On the relation between visual and auditory signs. In Selected Writings II, 338-344. Paris & The Hague: Mouton 1967.
Preziosi, Donald, Advantages and limitations of visual communication, in Krampen, M., ed. Visuelle Kommunikation und/oder verbale Kommunikation. Berlin: Olms Verlag, Hildesheim/Hochschule der Künste 1983, 1-34.
Saint-Martin, Fernande, Sémiologie du langage visuel. Québec: Presse de l ´Université du Québec 1987.
Sebeok, Thomas A., Contributions to the doctrine of signs. Bloomington: Indiana University Press.
Sonesson, Göran, Methods and models in pictorial semiotics. Lund: Institute of Art History 1988.
Sonesson, Göran, Pictorial concepts. Lund: Lund University Press 1989.
