Gestures in comics

A doubleshot of reviews:

Fein, Ofer, and Kasher, Asa. 1996. How to do Things with Words and Gestures in Comics. Journal of Pragmatics, Vol. 26.6. Dec. 1996. Pp. 793-808

This study looked at the role of gestures in comics (specifically, those in the European comic Asterix). The study had people interpret the meanings of both panels from the comics, and of photos where people took on similar poses. The backgrounds of the panels were erased, so there was no context for the gestures. In one part, they were asked to write possible dialogue for the gestures, and in another task they were given potential meanings and asked to assign them.

It concludes that gestures in comics are interpreted the same as ‘real life’ gestures, and that the meaning imbued in them comes from the ingesticular force (i.e. the intent of the expression) rather than the propositional content of the accompanying speech (in word balloons). One interesting tidbit noted that some people said the photos were actually harder to interpret than the comics panels (though the stats disputed this). If this were true, then it would support McCloud's insinuation that cartoony images are more "base" than realistic ones. I'd like to know the VL fluency of the subjects and whether people with more "comics" experience rated higher or lower in this regard.

Raecke Jochen. 1999. Using Comics as Data for Research into the Connection between Pointing Gestures and Deictics. In E. André, M. Poesio, and H. Rieser (eds). Proceedings of the Workshop on Deixis, Demonstration, and Deictic Belief at ESSLLI XI.

Uses comics to analyze the relationship between deictics and gestures in Serbo-Croation. His method codes a corpus of comics comparing the relations of the images' gestures to the conent of the speech balloons. He finds that pointing gestures by far dominate the gestures, and pointing gestures alone do not fulfill the meaning of the representations (i.e. multimodality is necessary). This isn't surprising, since pointing gestures are indexical, which means that they only indicate meaing in something else (the same way a pronoun refers to a different element for meaning).

Cupid Du Coeur

I've posted another segment of Permutations today. Here's some background on the piece "Cupid Du Coeur":

I greatly enjoyed the mixing of visual styles in this piece. Much of the computer work was done without much planning and with a large amount of trial and error to see what worked. The photo from the first page was found with some friends while looking through old papers in the recesses of my fraternity house in college. On the back was written "San Francisco 1895." It was such a striking shot that I figured I should use it for something, and so here I did.

I had toyed with the idea for this piece for a while before writing it, inspired partly from a chapter of Michael Crichton’s Travels. Its been one of my favorite books since I first read it when I was 12. Lately I've been musing how I've been taking on a lot of interesting experiences of my own, just as Crichton mused he was following in the footsteps of Arthur Conan Doyle. Its amusing to think that we can somehow be influenced enough by our favorite books to become similar to them... Of course, it could just be that that's why we liked them in the first place.

Musings on Time and Space

One of the papers I’m currently writing is about how “time” is understood across sequences of panels. Scott McCloud’s basic position on this is that “time = space,” so moving through space means that time is passing in graphic form. I have numerous problems with this, but more than anything it has made me question why this equation might be made.

I’ve been particularly into Benjamin Lee Whorf’s writings, who argued that the language we speak affects the way we think and perceive reality. A class on this "lingustic relativity" in college is what basically motivated me to study the relationships between linguistics and “comics” in the first place.

Whorf argues that the tense system found in most European languages is what (in part) has created our sense that time is a linear thing. Because we have a past, present, and future tense, it lines up all events in a row. He makes similar arguments to the extant that we create a sense of “space” out of our quantifiers and plural system that relates back to our understanding of time.

So, this begs the question: Will the people who speak a language without a tense system (like Hopi, as Whorf shows) have a different manner of conveying concepts graphically? Would their visual languages lack sequential events entirely in favor of something else more amenable to their spoken language? Would this make the visual langauge grammar dependent on the concepts from the spoken, or can both exist with different systems of conveying events?

This is one of the most fascinating possibilities for research I think: whether the language one speaks affects the language that one draws. In some ways, this is the question that a lot of my work is leading up to. All in good time I suppose...