Essay in The Public Journal of Semiotics

Way back in January, I had a piece published in the innaugural issue of the online Public Journal of Semiotics, yet kept forgetting to post about it. The essay there is an expanded version of my downloadable essay "A Visual Lexicon", which people seem to find as one of the most interesting of the papers posted (at least, that's what the vocal feedback says).

Naturally, I think the PJOS version of the paper is much better than the original, though the interface is a little funky since they're experimenting with electronic delivery beyond simple pdfs. Unfortunately, that means they lost all of my formatting, in addition to putting my images in places that don't always make sense for the flow of reading. Grr...argh.

Neverthless: an expanded essay for your reading pleasure.

Thoughts on a visual e/i-language

Big breath; prepare for a long post...

Most of the time, people think about languages as being "out there in the world" – consisting of a conventional list of words and rules that we all access. This leads to people commenting frequently things like "that's not a word," out of the belief that there is some external authority that dictates what is or isn't in a language.

In contrast, a cognitive approach to language looks at it as a phenomenon of individual's minds, and that the mutual conventionality of many minds creates the artiface of a system that is "out there" abstracted from those individuals. Several linguists have tried to delineate the relationship of a language to its speakers.

The first major one would have to be Ferdinand de Saussure, who made the breakdown between langue and parole. While parole describes a language in use as a dynamic social activity, langue was the notion of an abstract system of expression beyond its usage.

Noam Chomsky did Saussure one better though. Chomsky maintains that this system of expression is located squarely within a human mind/brain, what he calls an internal or i-language. In contrast, and external or e-language covers both of Saussure's terms: 1) external speech behavior and 2) the system as something in the world, abstracted as we call English or Chinese.

In reality, E-langauges are built out of the mutual intelligibility of people's i-languages, which often differ based on geography and community. A good example is an irregular derviational rule like the plural of "cactus." For a Southern Californian like me, the plural is the irregular "cacti," because it's common enough in daily life to be out of the ordinary. But for, say, a Bostonian who lives around me now and never interacts with them, the prickly stuff might be called "cactuses" using the plural "s" rule of English. Both are "right" in a cognitive sense, because the cognitive structure differs based on ecological context (most jargon is like this too).

An "e-language" distinction dislikes this, yet it's the reason that dialects exist at all. They are just degrees to which people's mutual intelligiblity of i-languages group in a graded way. The "r"-less Bostonians down the road from me certainly have different rules that they follow in their i-language than I do, but our systems are close enough that we understand each other.

And... all this is setting up some useful concepts for what I really want to talk about: drawing.

I've discussed previously that there is a cultural force of an "Art perspective" that affects the development of people's drawings skills in our culture. While they may be cognitively inclined towards imitation, the Art perspective guides them towards having an individualistic style and away from using the shared structures of a community.

OR, in Chomsky's terms... the Art perspective pushes people to have i-languages that (for the most part) don't build into an e-language. There are some exceptions to this of course, maily in conventional symbols like word balloons and speed lines.

Note the semiotic allowance here though: the symbolic aspects must be conventionalized and thus easily build to a shared structure that becomes more "out there" in the world abstracted from users. But, the broader iconicity of the system (that the signs resemble what they look like) allows for a degree of mutual intelligibility without mandated conventionality. This lets individuals' styles be inexplicably tied to their own i-languages.

We immediately recognize people's styles as belonging just to them, expressing their i-language, not tied to a broader visual e-language. And, people tend to get in a huff when other people "encroach" upon one's visual i-langauge. This is the taboo of copying another person's style, instead of being looked at as building or accessing a broader visual e-language.

A result of this is the belief that images have no explicit system behind them at all, since no e-language is built. Again, since drawings are iconic, and everyone's i-languages tend to differ so much, there is an appeal to perceptual knowledge as why we can understand images, instead of a specific cognitive system at work in graphic creation.

Just for the kiddies

This was a final thought that never got posted on the whole "Iconic Bias" kick. I started thinking about the old "Comics are for kids" misperception related to it all. On the one hand, I think we can all agree that this belief has come in part from the selection of genres and social contexts that propagated during the rise of the industry. But, on the other hand I think that a deeper issue might also be at work: the idea that pictures as a whole are somehow simple or lesser than (spoken) language.

We even see the derision of the graphic form in our speech, in idioms like, "Do you need me to draw a picture for you?" The phrase tacitly assumes that pictures are simpler than words, and hence drawing a picture will communicate the idea in a less complicated way. Now, this consideration of drawing could be considered a good thing ("Isn't it great how simple and understandable these complex ideas are presented in drawings!"), but here the tone usually remains derogatory towards graphics.

This "simplistic" perspective could also be related to the Iconic Bias issue: "If pictures just look like what they mean, how complex is that? …because we understand pictures just like we understand real life."

In this view, again, pictures are not conceptual (no mental system). Perhaps that's why people are always flabbergasted to hear that certain people or cultures have trouble understanding certain drawings or sequences of images (which does happen), as if it tears against the very fabric of their knowledge of drawings. The classic orientalist thing to do was blame the people, as if they were substandard or primitive, instead of (gasp!) seeing that their own system might be learned to a large degree and not as transparent as one would like to think.

Not Just Perception

So, it seems this Art vs. Language/Iconic Bias stuff is on the brain these days…

A great deal of research on "art" or looking at drawings has been situated in Perception studies of psychology, and I've on occasion bumped heads with various who have taken this track. Perception, as it pertains to visual things, studies how it is people's vision works: the psychology of vision. I can understand how it might fit with visual language studies, and it does, but only to the same extant that studies of hearing fit with spoken language.

This issue is related to the Veil of Iconicity issues that I've been writing about lately. Thinking that drawings should be situated in Perception fails to recognize that they can form their own system of communication, rather than simply being a router for the perceptual activity of vision.

Rather, my visual language theories seek to recognize that drawings are different. Drawings are not just "visual material" that is seen like everything else in the world (ahem, though the aparatus of vision might treat it the same). Drawings come from the mind, they are conceptual representations, and as such can't just be lumped in with all perceptual things that don't come from the mind without some distinction. Just like words in a language, they are part of a code. It just happens that this code often looks like what it means.

Iconic Bias, Part 3

So, last post I talked about how criticism of certain drawing styles stems from the fact that critics might speak a different dialect of visual language. The Art Perspective would have trouble recognizing the idea of graphic dialects because it sees drawing as a siphon for drawing "real life," which leads to the second reason why people might deride certain styles:

Reason 2: They see an issue of skill, not fluency

Rather than seeing graphic ability as a proficiency in a set of communal standards (fluency), people regard drawing as a "skill" that people are "better" or "worse" at. In the Art view, this skill is often determined by how accurate real proportions are held up. Judging representations based on iconicity gives an "objective" basis for claims of value.

Granted, the communal signs of the visual language community may not be prevalent as standardized signs like in spoken language (largely due to the forces brought this Art Perspective). However, within an individual author's style, they have consistencies and patterns. But, you really can just tell when someone is or isn't fluent, based on your own fluency.

I should also note "fluency" makes no judgment about the nature of the content. It purely has to do with reaching a certain degree of proficiency with drawing. Indeed, there are several very popular comics both on the web and not that are very well written and have great content, yet I'd say that their authors are less than wholly graphically fluent.

The fact of the matter is that Liefeld is undeniably graphically fluent: he has internally consistent and patterned ways of drawing and does so to a high degree of proficiency within that style. Sure, his proportions and anatomy may not be accurate to "real life," but this is only an issue if you believe that drawings should match real life, as opposed to patterned mental structures.

Aside from the issue of fluency levels, it is these same issues that lead people to think that certain drawings might be more "primitive" than others, like Egyptian drawings or cave paintings. Appealing to iconicity lets there be a scale of "progress" relating different styles to each other (like the notion "we've progressed so much in drawing since those old days"). Systems using point perspective and shading are believed to be "superior" to those that are flat or fixed to a certain angle-of-viewpoint, because the former is more iconic. A Language view doesn't allow this: all systems are equal, only they do things in different ways.

And finally, a thought to sum up the way that Art can (and does) fit with a Language view: Language (including visual language) is what a structure is, Art is what you can do with it.

Iconic Bias, Part 2

As I discussed in my previous post, there is an Art Perspective belief that people should learn to draw from "real life" and not from copying others. In that post, I tried to tease out some of the underlying assumptions of this belief. Here, I'd like to point out how it leads to derision of other approaches (as demonstrated in the quotes from the last posts).

(Caveat: Just to be clear, and to avoid misperceptions, I don't have a problem with people drawing realistically. I'm just trying to examine the underlying assumptions that motivate perceptions of drawing. And, I should also add, that I too am a product of the culture of Iconic Bias, as it did and still does inform many of my graphic decisions.)

There are two reasons those who hold this Iconic Bias might think that certain styles, like say Rob Liefeld's drawings or the manga style, are substandard:

Reason 1: They speak a different graphic language/dialect

People will rag on certain styles mainly because they belong to a different graphic dialect. People who aren't comfortable with the early 90s Image style or manga style (or many others) as their visual language end up making fun of it or calling it substandard. This is similar to the treatment of African American Vernacular English (popularly known as Ebonics) in America. Some speakers of Standard English end up thinking that speakers of African American Vernacular English are somehow stupid or have less skill in language, when really the fact is that they speak a different dialect of English that has its own patterns and consistencies. It's not "lesser" its just different.

Of course, the Iconic Bias has an easy time making these judgments, because it doesn't allow for there to even be such a notion as graphic dialects: drawings are valued on their relations to "real life perception," not mental patterns. The sheer recognition of drawing styles as cohesive systems runs against the free-for-all in how "each individual interprets real life."

More coming in the next post…

Iconic Bias, Part 1.5

Here's another subtle example of this Iconic Bias that I discussed in my earlier post. From a recent new interview with Scott McCloud, the interviewer asked:

"What’s hard for you to draw?"

Can you see the subtle bias here? Give it some thought…

It assumes that the capacity for drawing is about "drawing things in the world" — and drawing them as their supposed to look — as opposed to drawing being a capacity for expressing concepts, which just happen to look like things out "in the world." Let me rephrase the question: "What [things out there in the world are] hard for you to draw?"

In contrast to a Language approach, imagine asking someone, "What words are hard for you to pronounce?" Scott's answer is also illustrative, because it invokes the need for reference photos, tied to that same perspective (if its not in the mind, I need to reference "out there in the world").

Snark I can't hold back: Why is this question even there? I mean, really, this is Scott McCloud, the guy who ushered in deep thinking about comics. You really want to waste a question asking him what he doesn't draw well? Puh-lease.

OK, I swear, next post on this I'll get into some of the results of this Iconic Bias.

Iconic Bias, Part 1

I've tried to point out in some of my writings that there is an "Art" perspective that dominates the usage of graphic creation in our culture. Its primary emphasis is for innovation and individuality in styles in contrast to a "Language" perspective that seems communality and shared signs.

Regarding learning, the "Art" view has two emphases:

1. Developing one's own style apart from everyone else (i.e. "don't copy other people's drawings")
2. Learning to draw by imitating real life

I'd like to take aim at #2 in this post, since I think that it carries more assumptions with regard to it. #1 may be something that is emphasized, but its hard not to get influenced by other sources anyways, and the prevalence of "house styles" (Marvel house style, etc) or "cultural styles" (manga style) easily shows how group consistency does happen. Of course the assumptions under #2 motivate the "anti-style" tendencies of #1 as well.

The basic assumption underlying #2, that people should learn to draw by imitating real life, is related to what I call the "Veil of Iconicity." This outlook treats graphic creation (drawing) as something that is not rooted conceptually. The graphic form is merely a siphon through which "real" (looking) things are represented. And, if not done directly through "life drawing," then that mode is what should at least inform a person's drawing "from their imagination".

Now, several people have tried to tell me that "drawing from life" is no longer an emphasis of Art, but I've found these quotes recently on a variety of comic message boards:

"I hate manga/anime. You know why? They are all hacks. It doesn't take any talent to draw round heads and big eyes. It takes a lot of skill to draw accurately proportioned figures that are anatomically correct (traditional line art)."

"Comic books, whether drawn by pinheads like Liefeld or superb artisans like Gil Kane, are a terrible training ground for drawing - even if you intend to learn how to draw comics. All you're picking up the the graphic shorthand the artist has developed to represent figures and environments so he can draw from his imagination. It's always a stylistic distortion, a poor substitute for life drawing or copying photos."

"...photos will build your skills a lot better than copying someone else's ink lines."

This is exactly the Art perspective I'm talking about.

Again, this perspective leaves out the role of the mind in relation to drawing. Patterned ways of drawing, as stored in long-term memory (your "imagination," as if it were a bad thing), are made to seem less valid than just rerouting perception through the graphic form. Sure, by definition an "iconic" sign "gets its meaning by resembling what it represents." However, that doesn't mean that such signs must exhibit this to the maximal degree, all the time (if at all), and without accepting that such signs must come from the human mind.

I'll continue this with the next post, delving more into some of the results of this belief.

Problems with Closure, part 3

In my last post I pointed out that pictures are not believed to have constraints on them, and that the mind must place constraints on any sort of understanding:

Assumption #2: The Veil of Iconicity

This assumption is that pictures are “out there in the world,” not learned information, and thus not mental phenomena. McCloud shows this underlying belief by stating:

“Pictures are received information. We need no formal education to “get the message.” The message is instantaneous. Writing is perceived information. It takes time and specialized knowledge to decode the abstract symbols of language.” (p. 49)

This belief is formed because images are most often iconic, meaning that they derive their meaning through resemblance to what they reference. A picture of a person is known to refer to a person because we know what people look like in the world. Note, there are three parts to this equation: the picture of a person, people in the world, and the concept of people in our minds.

However, just because they look like what they mean, it doesn’t mean that pictures aren’t conceptual information. Through this resemblance, we forget that it actually requires a mind to understand these images, and thereby discount its contribution to understanding. Images just seem like what we experience in the world: we don’t seem to need any special understanding to know the world, so thus we don’t need special understanding to know images.

Upon closer reflection, this is somewhat of a ridiculous mistake. If I draw a picture, how can it not be connected to my mental understandings? It came out from my mind, why wouldn’t its reception need to go through my mind too!? I had to learn how to draw, doesn’t that mean I had to learn how to understand drawings too!?

Considerable studies have shown that the understanding of images is clearly not so transparent. Often, this is found in native communities like Australian or Amazon aborigines who couldn’t/can’t understand aspects of "Western” representation. In the past, this was haughtily used to justify their intelligence as "primitive" compared to Ameri-Europeans. Really, this is just a case of not having fluency in the conventionality of a graphic system (natives for the Western system(s), and Westerners for the native systems). Science is rife with these sorts of examples treating the world “objectively” while really being unable to see beyond the petri dish that oneself is standing in.

Because images look like what they represent, we gloss over the mental component for understanding them, and in turn is misplaced for sequential images. I’ll take this up in my next post.

Problems with Closure: Part 1, Part 2, Part 4

Introspection and the haze of definitions

In one of my linguistics classes last year, my professor claimed that the arbitrariness of language was noted as early as 2500 years ago in China, by the philosopher Laozi. For those who don’t know, arbitrariness means that the sounds of language don’t have any direct connection to what they mean, they are purely conventional associations. Usually, this insight is attributed to a linguist named Saussure, from the early 1900s, which gave rise to “structuralism” and “semiology.”

Laozi did notice that language was arbitrary, but it certainly wasn’t the intent of his observations. Really, he was pointing out that not only did words lack a connection to their meaning, but because of it they were a hindrance in the search for Truth. This is the first thing that sparked my interest in language and cognition, albeit in a roundabout way, back in high school.

I’ve been noticing lately about how this mentality really underlies my work though. For many of the things I discuss, it’s not so much a matter of defining things clearly as it is breaking down those concepts. My non-definition of “comics”: Comics is not a precise combination of text and image values, but rather a cultural object, a sub-culture, a community, a genre, etc. that exist in society.

I give a similar treatment to “Language.” Rather than saying that “language” is a thing that people can concretely put their finger on with one defining element, it is instead a manifestation of several features that include: A sense modality, sequence, meaningful reference, combinatoriality, communicative use, social usage, a social identity,… along with several others.

In both of these cases, the “definition” comes out of an aggregation of a variety of elements. But can you really say that a conglomeration of parts is really a whole “thing”? Buddhist thought would say “no” (which it does quite powerfully to the notion of a “self”). The definitions are fully understandable, yet empty.

This is also fairly apparent in my definitions of “writing” and “drawing,” which I explored in my (rather long) MA thesis. The gist was that these notions are contingent upon the systems that we use and their mapping along a large triangular map of signs. The whole triangle is the “Truth,” but nobody accesses in full. They have to access it through the portions that they cut up.

The understanding of the parts and the “why” is really what I’m after, and pushing through the illusion that the words create a concrete concept is the only way to get there.