Vocab gaps

This interesting and quite fun essay/post reviews the book Reading Comics and ponders the definition of "comics" and some other terminological issues. Its biggest query is 'what is the word for the act of making a comic... the producing of sequential images bit?' (paraphrased).

I agree there's somewhat of a gap in vocabulary, but think that its partially symptomatic of a larger issue: not recognizing that drawings (especially in sequence) constitute a system of communication. That is, we lack the recognition that drawn works use a set of mental patterns the same way that, say, English does.

We in America speak in English.
We in America draw in ___?___.

Note that we also have many words describing the avenue or subroutine for picture-making: painting, sketching, penciling, inking, coloring, etc. But, no word for one actually does with those media besides the uninformative "drawing," which inadequately covers the dynamic process of creating sequences of images.

In a previous thesis of mine (¡Eye [heart] græfIk Semiosis!), I argued that vocab in a language for graphic creation partially relies on how that culture's graphic systems are structured. In Western representation, we have a dominantly sound-based "writing system" with a preference for highly "realistic" drawings — and our words for these things are very different: "Writing" versus "Drawing." Contrast this with Japan, who use a wide range of meaning/sound values in their "writing", while using much less realistic representations — and they use a single word for both concepts (kaku: かく), while using two separate Chinese characters to distinguish uses (writing: 書く, drawing: 描く).

So, in our case of English, what we're left with is a gap in vocabulary, for those representations that run a middle ground between our prototypical conceptions. Anyone who reads this blog should know my answer: The missing graphic system is what I call "visual language" (when done with images in sequence).

We in America speak in English.
We in America draw/write in American Visual Language.

If this is language, then we might as well just adopt language related vocabulary. I have no problem with the idea that I write in pictures (as well as words), and I doubt many others do either.

And, as I've argued before, recognizing that such a system exists (visual language) is the first step towards defining (or rather, un-defining) "comics." By taking away formal definitions that rely on the "comics=visual language" equation, "comics" can be defined as what we've always known it to be: socio-cultural objects (books, strips) in which a visual language is written, often pertaining to particular genres, and a culture surrounding them.

Finally, many have complained that debating vocabulary is a waste of time. But, something like this has realistic applications. Take for instance pushing the notion of visual language through the idea that sequential images are "written." Where would you learn such a thing?

You learn to write in a writing/language class, and such would be the case with visual language too. You learn to write in pictures in a writing class, not an art class — bringing it directly into the fold for education and development in a very practical and communication-driven way.

Dunkin' pictograms

Speaking of pictographic writing systems that will never be universal, Fabricari sends along this logo from Dunkin' Donuts:



I've been seeing that logo for over a year now, and never quite got that the images were supposed to be a pictographic sentence until a few weeks ago! I had just thought that it was a bunch of pretty pictures. Of course, given that I rarely if ever go to Dunkin' Donuts, I never really put in the effort to try and decode it.

Though, while we're here, I might as well use it as an example as to why pictographs fail as universal systems. Outright, as I mentioned in my last post, the grammar here completely mimics that of English. Of course, that was the intent in this case since its a slogan, which is why there is four units for four words. But, notice also that this conversion makes a very important decision: it chooses to transcribe "America runs on DD" as a verbal-->visual mapping, rather than siphoning the concept behind the idea into the visual modality to then adapt to its own traits. Moving on...

Let's start with nouns. First off, the DD is only understandable if you know the association to the company. The map of America could be read as "map" or some such, but is even more interesting since it is a metonymy. It uses "America" to mean "the people in America."

The verb "run" nicely shows how you can't visually show an action without also showing an object. It's tough to show "run" without also showing the "runner." Verbal grammar (by virtue of its symbolic nature) likes to divide these pieces into [ACTOR]-[ACTION]. In visual grammar this division doesn't work as well (being iconic, not symbolic), instead becoming [ACTOR:state1]-[ACTOR:state2], where "state2" shows the fruition of the action.

Finally, the preposition "on" isn't visually converted at all. I find this to be particularly telling, since it immediately shows the English context. I imagine also that the makers of the logo struggled with it, since this usage of "on" is not the spatial kind ("on top of") but is tied to the verb.

In fact, the interpretation of "run" as an action here (like "run down the street") is wholly off, since they don't mean that Americans "use their legs to run on top of Dunkin' Donuts." Rather, they are using a construction "run on" (arguably not two units) that means roughly "to be powered by." The "person running" image then becomes a "double rebus" --> first mapping the sound pattern to the image, then the image's literal meaning to it's "metaphorical" meaning.

To come back around to my initial statement about mimicing English grammar, this actually can't even do that since the slogan doesn't use concrete elements. A literal reading of this ends up being totally bizarre (bracketed by panel, italics adding clarifying info):

[The country of AMERICA][uses its legs to RUN][ON top of][DUNKIN' DONUTS]

SO... this example only goes to reinforce how hard it is to "accurately" map verbal expressions to visual signs — both individual signs and grammatical sequences — especially when it involves metonymic and metaphorical expressions (add those semantic aspects to the list then). While English speakers might be able to figure this one out, can anyone possibly imagine this working on a global scale?

Sorry, there will never be a universal writing system

Jochen Gos emailed me awhile back with news of her new book Icon-Typing and website Icon-language which is a font that creates pictograms when a person types in their own language's sentences. Unfortunately, it's taken me a rather long time to finish this post about it.

While it's a fun idea, and reminiscent of other projects of the same nature, I should say that I do not consider any of these approaches to be "real" visual languages. And, perhaps dissapointing to some, I don't consider any enterprise that strives for a "universal pictographic writing system" to be a fruitful endeavor for multiple reasons.

I do think that the idea of a pictographic language stems from a deeply felt intuition that there is indeed a visual modality language, but cannot reconcile it with the assumptions our culture makes about graphic signs and language. Among the problems holding back these types of endeavors:

1. Semiotics - There is a general misunderstanding, both in and outside of academia, that there are only two types of graphic signs: those that map to "ideas" – mistakenly like Chinese or pictures – (ideographs) and those that map to sounds - like alphabets (phonographs). This is a gross over generalization that I attacked in my thesis ¡Eye græfIk Semiosis!, trying to show that there is a much more graded system of expression at work here.

2. Grammar - All "univeral" pictographic languages must piggyback on the grammar of existing verbal languages. Verbal languages are vastly different in their grammars so much so that no possible universality could ever be reached. There is a deeper reason for this though: dividing pictograms into linearized individuated units denies the grammatical qualities inherent to the actual visual form. A grammar must arise up out of the signs themselves, not be imposed on signs from the outside (see again my paper ¡Eye græfIk Semiosis!). An Example: A pictographic system might use a marker to show plurality paired with an icon. So, a pictographic symbol might express "men" like [MAN PICTURE]+[PLURAL MARKER] (not necessarily linearly)as opposed to just showing multiple men couched in an environmental setting. In one, the concept arises directly from the sign, in the other it's latching onto the way that verbal languages do it.

3. Morphology - Believing that pictograms from all languages can easily substitute in for the words of a language presupposes that all languages have "word" units like English. What would this do for a polysynthetic language like Greenlandic or Dene that chunks multiple meaningful units into a whole syntactic unit (i.e. a single "verb" in these languages might have the same quantity of information as an English sentence). Chunking up visual pieces like this betrays a bias for how European languages break up units, not all languages.

4. Linguistic relativity - To believe a universal writing system is possible assumes that all languages structure concepts the same way. This is simply not true. The aforementioned Dene has several forms of the same verb meaning, each one reflecting a different type of object (for instance, the verb "give" changes based on the texture and character of what is being given – whether its animate, granular, squishy, flat, etc). Less "exotic" differences are like that Japanese lacks plurals and definitive articles. Plus, given how widely languages differ, which morphemes would be worthy of visual conversion? Do you convert all of them (including things like transitivity or definiteness?) or just those most "visually relevant"?... and how would that not be a subjective decision based on the preferences of the speaker of a particular language? A speaker of English might be more inclined to need particular features (like plural) that differ from a Japanese person (like politeness). A universal writing system assumes that concepts are universal to express and do so in similar ways: a big assumption.

5. Transparency and Iconic Bias - There is a general belief that Iconic signs that look like what they represent are wholly transparent in their meaning and non-culturally relative in and of themselves. This is also untrue. While most all cultures seem to be able to decode what "realistic" drawings represent, they often are subject to entrenched variability. For instance, Australian drawing systems usually feature aerial viewpoints, which can lead to odd interpretations of lateral viewpoint representations. Furthermore, simplfying the "concept bundle" of images down to a one-to-one concept-to-sign ratio is extremely difficult.

Finally, the biggest issue here is that the visual form is implicitly considered as lesser than the verbal. It uses the verbal form to be translated into the visual without stopping to think that the visual form might already have its own visual language system naturally that doesn't need a verbal connection at all. Writing systems themselves are not a bad thing, but as an importation of one expressive modality into another they will always be subject to the constraints of the imported system.

A real, natural Visual Language should be able to stand alone without needing such connections to lean on, and would reflect the diversity of different cultures as well.

License plate linguistics

Here's a little fun with writing and ambiguity. A few nights ago I saw a license plate with some friends that read:

OYABABY

Oddly enough, I parsed this first as being "Oya Baby" — drawing from my Japanese knowledge. In Japanese "Oya" is "parent", and there is a dish served with chicken and eggs over rice called "oyako don": Parent (oya) and child (ko). So, my first thought was that this plate was a Japanese oriented person making some sort of strange reference like that.

Then I hit upon another interpretation... Maybe this person was Jewish, with a nice "Oy! A Baby!"

Of course, right as I said this aloud, my friend proclaimed what seemed like it should have been obvious: "O Ya Baby!"

It seems my knowledge of Japanese and Jewish exclamations trumps my coolness.

What I particularly liked about this exchange was that each of the three parsings covered all the possible combinations for those letters:

OYA
OY__A
O__YA

And, with each parsing, the polysemous meaning of "Baby" took on a different meaning. For the first two, it was an infant, while for the second it was a familiar term of endearment (for lack of a better descriptor). (Though, you could flip the Baby meanings too: "O Ya Infant!"). It's a nice demonstration of how ambiguous written representations can be without an express phonemic link.

Of course, that license plate still isn't even close to my favorite plate from Berkeley: GRRARGH

Codex Boturini

My old friend John Jacobson passes along this link to the final pages of the Mexica Aztecs’ Codex Boturini which shows some very interesting text/sequential image combinations. It’s definitely worth reading through the Brief Readings of each page, and at least perusing the essay.



One of the things I find interesting about it is its use of footprints to provide the "path" of the gods, which helps "unify the design of the manuscript." Structurally, this is interesting because the footprints retain an aerial view while the rest of the images feature a lateral viewpoint. This is reminiscent of some of the drawings by Arrernte children who unite the aerial view of sand drawings with the lateral view of Western representation.

It also seems that the reading of the manuscripts is somewhat as a mnemonic – not fully a visual narrative that draws its meaning from the properties of the graphics alone, joined by the meaning of the words. Rather, it lies on the cusp area of my CMGS where the drawings represent mnemonic signs for "set of concepts that could be verbally formulated in a number of different ways," used as a supplement to oral performance.

This type of visual concepts is an interesting feature of many older systems (including the Tibeto-Burman Naxi as well as several others). It’s kind of a halfway usage of the visuals without, in my estimation as yet, fully becoming a visual language, while also not using the transcription system as its own stand-alone system of "writing."

Reposted 8/11 with fixed links