My "Homopholganger"

Huzzah! Today is the sixth year this website has been online! If I remember correctly, I posted everything online while the Lakers were in the playoffs about to go to the championships... and lo and behold the past is repeating! (yes, I'm a Laker fan... which will certainly be interesting living in Boston as they move on to play the Celtics in the Finals)

So, here's a semi-research-related story to commemorate the occasion. The Tufts Psych department (to which I'm a grad student) is hosting a conference this weekend, and one of the featured speakers is a psychologist named...Neal Cohen! (no relation)

Naturally, I thought it would be hilarious and awkward to meet him. The first day of the conference I had turned it into a scavenger hunt, with several faculty and other grad students all on the lookout for him. We came up with a great portmanteau word to describe someone who shares the same name as you: your "homopholganger." By the time I arrived Friday, I was getting asked over and over if I'd met him yet.

I actually did end up talking to him shortly after his own presentation, and hilarity ensued! Even cooler, I started making connections between some of his work on the hippocampus to things I'm finding in visual grammar. Naturally, I proposed a collaboration... He thought the ideas were pretty cool, so, who knows, perhaps in the next few years we'll see the fantastic byline: By Neil Cohn and Neal Cohen.

Panel Time!

1.5 seconds per panel. That's how long it takes on average for wordless panels to be read.

I recently completed a very exciting study that asked people to read four-panel comic strips one panel at a time. In this "Self-Paced Reading" task, they see four boxes on the screen, and with each button press a subsequent panel appears in the sequence. Only one panel is shown on the screen at a time.

While they do this, the computer records how long it takes them to move from each panel to the next. By manipulating the strip in various ways, I'm able to tell if certain manipulations have a greater impact on the reading from the original "normal" sequence.

I'm not going to go into the intricacies of the experiment (you can wait for the write up for that), but I thought I'd share some tidbits.

For instance, for "normal" wordless four panel Peanuts strips, it takes a person on average 1.5 seconds per panel. The first panel is usually read relatively slow (1.7), the second panel is fastest (1.3), then third is slightly less fast (1.5), then fourth is back to where the first was (1.7).

An interesting side note about this: since all these panels were the same size, yet were read at different speeds, it might imply a rejection of McCloud's claim that panel sizes affect reading time (which thereby somehow affects narrative time). However, this experiment didn't test that specifically, and no alternations in panel sizes nor layouts were given either. So, I can't say anything conclusive about that.

This was a very exciting project for me to complete, because it marks the first time I've (anyone has?) looked at data like this for how people understand sequential images.

The Invisible Middle Ground

A friend of mine and I had a really interesting discussion the other day about how difficult it is for a reasonable, mediating, middle-ground type of theory to survive in the scientific landscape. Within linguistics, there are a lot of debates I think a middle ground position serves the most efficient explanation.

However, there are several things holding back the success of such views. For one thing, extremes are usually louder. People are more vehement when defending/attacking a radical viewpoint. A mediating voice may simply be passed over as "not getting the whole picture" by the extremist views, instead of as a "reasonable middle ground." Both sides then view the middle ground as too much like the other side to be accepted.

As my friend pointed out, it's also the nature of statistics and experimental design to not support middle ground viewpoints. Outside of the rarely used Bayesian stats, hypothesis testing forces a binary distinction of confirmed or denied. That is, scientific methods promote extremist viewpoints.

If a hypothesis is tested and reveals a confirming answer, it is usually taken to mean that it is right and that opposing views are not — even if opposing views also receive confirming experimentation. What ensues is usually an unreconcilable clash of arguments, where neither side can see the validity of the other. Can't two rights exist at the same time?

Moreover, competing views often legitimately belong to differing paradigms of thinking. New paradigms are often asking new and different questions than the old ones, and thus providing new answers. Often, this involves chucking the old paradigm. However, do paradigm shifts have to throw out both the baby and the bathwater?

Perhaps Sinfest has the right idea...

Odds and Ends

Between school, conferences, and life, my time has become quite precious lately. A few days ago my BioPsychology professor said that we'll be dissecting sheep brains in class. That should be interesting.

I've finally gotten a chance to analyze a bunch of the data that has come in from my Peanuts experiment and things are looking very cool. I haven't yet run the actual statistics, but some initial graphing of results show signs that my predictions about visual grammar may be borne out. Time to start crunching numbers...

Bob Weber passes along a link to his blog where he posts kids' drawings and gives encouraging comments. Its a nice interactive idea. What I'm struck by most is the huge range in ability for various ages in the posts he has there.

Finally, if today is your primary, be sure to get out and vote!

New Year, Old Projects

Usually when I'm on break, I end up blogging quite a bit. I'm surprised I haven't been as active online lately. I have, however, been working on several projects...

I'm very very close to having my paper about how people navigate through page layouts done, and I'm getting really excited about how the theory in it has developed. Hopefully it will make its downloadable appearance soon.

Over break I finished my review of Groensteen's The System of Comics, which will hopefully emerge in a web-based journal sometime soon. I'm still waiting to hear back about that.

I'm also getting closer in my analysis of the Peanuts experiment data, and making my way towards the final leg of that project. I'm already looking down the barrel of my next project which will create novel strips out of panels from various Peanuts strips. That should be a lot of fun, and a whole new round of interesting.

Finally, tomorrow I'll be recording a podcast about my visual language research to help promote the VizThink Conference I'm speaking at in a few weeks in San Francisco. Once it goes live I'll be sure to post it/link to it.

Some Peanuts patterns

It's the end of the semester, and as usual things are crazy. I've finally got my Peanuts experiment up and running, which means people are coming in to participate. A lot of people. The experiment lasts an hour, and between last week and this week, I'll run 24 subjects, which means I'll have lots of data to pour over during winter break.

In the meantime, I also finished coding several strips from Peanuts, and have found several interesting things. For this experiment, I culled 180 strips from the first two Complete Peanuts volumes (kindly donated by Fantagraphics), which were either silent or I altered to become silent. I then coded them all panel-by-panel. That's 720 panels, and yes, it took me all semester.

So, what did I find in my sample? Well, there is some interesting stuff...

Most of what I coded for has to do with narrative structure, or what I would call visual grammar. I'm hoping my redone terminology is transparent enough to follow here.

Out of 180 strips, 140 of them (78%) used "Establishers" to set up information in the first panel. Conversely, 123 of them (68%) finished with a "Release" where the tension of the narrative dissipates. 135 (75%) also use an "Initial" as the second panel, which initiates the actions of the strip. 112 (60%) finish the strip with a "Peak" — the height of narrative tension.

Of 180 strips, 50% (90) use the overall structure of "Establisher-Initial-Peak-Release." The next highest isn't even close, with only 13 strips using the pattern "Establisher-Initial-Initial-Peak."

The "E-I-P-R" pattern is what I think of as the canonical narrative arc (which on a macro scale resembles the traditional "narrative arc" of plotlines). All this aligns even more interestingly to coding I did of event structures for each characters' actions, but describing all that here might be a little overkill.

Just as a reminder, this is a very specific sample of strips and shouldn't be construed as making any sort of claims overall about Peanuts. Nonetheless, its fun to see what info the strips alone hold. Now I'm even more excited to see what the results of my study show about people's behavior in relation to these coded predictions.

Projects galore

Wow, three blog posts in three days, and I'm not even on winter break yet! Lots of stuff has been going on.

First off, my buddy Alexander alerts me to a strip over at the Comics Reporter that somehow slipped my eye when I was over there this morning. It's a good treatment on graphic fluency and why being able to understand comics involves cognitive skills — that some just don't have.

Anyhow... I'm extremely close to being able to start running subjects for my Peanuts comics experiment. This has taken me waaaaay longer to get up and running than I expected, so it'll be good to finally start getting some data. In hopes of not running into this problem again, I'm already planning to set up my next experiment, which will use newly created sequences made of various panels from Peanuts strips. I'll talk more about it once things get closer.

Additionally, there's a good likelihood I'll have a major new paper posted online by next week. Unfortunately, its not the paper on page layout (which is still undergoing editing... hopefully over winter break), but it is another paper I did over the summer that's due to be published in a collection next year.

Oh, and I have to do a book review for a class I'm in, so it looks like my critique of Groensteen's System of Comics is finally going to be finished sometime soon.

Status report

School is back in full swing, which means my time to devote to other projects (like blogging) has received a decimating blast. I'm still planning to post my research project on page layouts sometime soon — I just need to find the time to make some edits at this point.

On the good side though is that I have multiple projects in the works. My experiment using Peanuts strips is well underway in the coding and preparation of stimuli, and should hopefully be rolling with subjects in a month or so.

I'm also working out a coding schema to analyze the various donated books I've got on my shelves. Hopefully that will tell us some interesting things about the ways different populations encode information in their comics. Most excitingly, I have some students interested in working on these projects with me, which means the potential to get a whole lot more done that isn't just reliant on my time constraints. (yay!)

And, having just experienced a rather boring summer rehabbing my hip from surgery, I'm now trying to plan ahead to next year by designing a "Cognition of Comics" course. We'll see if it gets picked up for the summer school, but if so, it should be a lot of fun.

Also in the news, I'm now slated to be a fascilitator at the VizThink Conference in San Francisco in January. There's an interesting and diverse line-up of speakers, so it should be quite the event.

Oh, and I have a meeting with Noam Chomsky in a week. That should be interesting.

Video of Ray Jackendoff

It's not quite topical to the things I usually talk about on this blog, but my advisor recently gave a talk at Google that has now been posted to video:

Jackendoff's talk on "The Peculiar Logic of Value" centers on how humans conceptualize systems of value. He hypothesizes that value is conceptualized as an abstract property attributed to objects, persons, and actions. There are several distinct types of value, including Affective value (does it feel good or bad?); Utility (is it good for me?); Prowess (is so-and-so good at doing such-and-such); Normative value (is it good of so-and-so to do such-and-such?); Personal Normative value (is so-and-so a good person?); and Esteem (does so-and-so have a good reputation?). Each of these kinds of value plays a different role in the ecology of the value system.

Blasphemy!

As I've mentioned before, I'm currently designing a series of experiments that will use Peanuts strips for stimuli (kindly donated by Fantagraphics). Lately, I've been doing the arduous process of scanning them in and coding them.

Since I'm only trying to look at how sequences of images create meaning, and because the inclusion of text changes things, I've been trying to only use silent strips. Luckily, Peanuts has a lot of those. Additionally, I've also been using strips that have minimal text or could be (gasp!) manipulated to have no text.

I've been taking these strips and deleting the text, then touching up the mouths, etc. so that they work alright silently. It's been quite fun to make sure it all looks like Schulz's style. On the one hand I keep thinking "this is so cool!" and on the other I can't help but think, "Blasphemy!" for defiling the originals.

Most of the time I cut and paste from one panel to another, like putting a frown over an open mouth. In most cases, not much at all is lost in the meaning. It's usually like erasing a word balloon saying "Good Grief!" to just Charlie Brown frowning. The meaning (and humor) stays pretty much the same. I think this is actually a testament to how great Schulz's visual humor was, that I could go in and muck with things, yet the original meanings still come through.

Now... once I start swapping panels around or creating new strips out of panels from a variety of strips, then that might be another story...

(Amusingly, my father actually asked me if there were ethical issues with doing that... my advisor just thought it was pretty amusing. I'd have to think there are less ethical issues involved with retouching comic strips than experimenting on animals like the lab next door, but if I start seeing protests out front I'll reconsider).

I'm a winner!

Apparently, I've won an award. I just got an email saying that my "paper, The Grammar of Comics, was selected as the winner of the M. Thomas Inge Award for Comics Scholarship for the Comic Art & Comics Area at the 2006 Popular Culture Association conference in Boston."

So... yay!

Yowza!

As I've probably mentioned before, I'm currently building a corpus of comics to be used in research related projects. I've had some donations from companies already, and Dark Horse just about doubled my collection in one fell swoop! They are thanked tremendously (as are everyone else who's donated). I'm still interested in expanding the collection, especially with international books, so if you can help out with that, shoot me an email.

The ultimate goal, which we're inching towards implementing, is to create a massvie database of books. We will code various properties of the books, then be able to mine the data to come up with lots of interesting statistics about the structure of comics — cross-listed by country, genre, company, etc. This can tell us about the trends and patterns in visual languages of the world. Plus, all the information will also store coding information about the panels, strips, and books that I will be using in experimentation.

Hopefully I'll be able to get this set up to start having data entered by the Fall (most likely by enterprising undergrads... anyone out there go to Tufts or nearby schools and want to do research for course credit??).

Don't!

My department here at Tufts is hosting a big linguistics/psychology conference this weekend that everybody is going crazy organizing and getting ready for. I'm not really doing that much though, since I'm not presenting or participating that much (I'm actually heading to Santa Barbara on the weekend to watch my brother graduate college!).

Anyhow, my advisor asked me to draw a few comics for his upcoming talk and book so I figured I'd share. I'm not sure what the full discussion will be, but he said these will be used to talk about how sometimes reference occurs in places outside of sentences themselves:

Coercion... of meaning!

Today I gave my big first year project presentation to the psychology department. From what everyone has said, it went very well. Of course, the project itself is still underway, and I will be running several more subjects in the lab, while probably continuing the study as a whole throught the summer. This shot is of me and my advisors from afterwards. (R to L: Ray Jackendoff, Phil Holcomb, Me, Gina Kuperberg):

As I've mentioned before, my talk involved looking at the "Event Related Potentials" or ERPs involved with processing a certain type of linguistic phenomenon called "semantic coercion." ERPs are a measure of the electrical activity of the brain. We don't get a good fix on specific brain areas that are at work, like in fMRI, but we do get very detailed analysis of the time course of events and certain waveforms do seem to indicate types of brain functioning in contrast to each other. We measure this electrical activity by sticking a cap of electrodes on people and feeding the signals into a computer, which then averages out the noise over several subjects and trials to give a smooth wave for time locked events. Here's me in the cap...

So, I looked at these brain waves for semantic coercion, which involves the extraction of "hidden" meaning from sentences like The chef finished the chicken before the main course. Someone can't literally "finish a chicken," they have to finish doing some action with it, like cooking. Since the event isn't stated outright, it's said to be "coerced" from the combination of the verb "finished" and the direct object "the chicken." Here's a waveform from one of the sites on that cap that I got in the experiment:

While this is interesting as a linguistic phenomenon, I think it's really just a warm up for more comic related studies. Since I couldn't resist, I even opened my talk by showing this strip:

Now, if you look carefully, coercion happens here too. We never see the event of Snoopy catching the ball, yet we know the event happens based on the information provided by the other panels. In addition to other things, coercion is perhaps one of the things that McCloud was trying to get at with his notion of "closure." In many ways, coercion here is an invisible meaning that is created out of the visible components of the graphic sequence. Graphically, it's the stuff that happens "out of view" of the panels. The problem is that McCloud extended this to the (linear) relationships between all image sequences, which just doesn't work.

So, if I do find anything fairly robust in the ERPs for verbal coercion, perhaps a study of visual language coercion could be on the horizon as well? Or perhaps a theoretical paper first...

Reviews coming soon (hopefully)

I recently got Endnote for my computer, a program that manages citations and automatically generates bibliographies (a dream program for someone like me who writes a gazillion essays). I had previously been writing little reviews for books and papers into a text file, but now I can put them in Endnote nicely organized. As I'm now going through and entering in all these references, I'm sure to be writing lots more reviews. So, be on the lookout for lots more summaries of research papers, coming soon!

Well, to be honest, not too soon... I'm currently in the midst of massive projects, presentations, and exams for the end of the school semester. Hopefully I can do more blog stuff and personal writing in a few weeks once things die down a bit (well, relatively speaking at least... after that I have hip surgery, summer school, and even more research...).

Speaking fun

As I mentioned in a previous post, I'm going to be speaking next week at the Popular Cultural Association National Conference here in Boston next week. If you're interested in attending, I recommend checking out their site. I'll be presenting once again about my work on visual language grammar.

Then, I'm almost more excited to say, that I'll be teaching my stuff to the Syntactic Theory class that I'm a Teacher's Assistant for here at Tufts since my professor is going to be out of town that day. My advisor is the teacher of the class, and since he pretty much helped invent modern syntactic and semantic theory, its been a thrill just being in the class let alone getting to teach a lecture or being the TA.

The course so far has aimed less at teaching the students how to do a particular theory of syntax (though it has done that a bit, advocating my advisor's new theory of Simpler Syntax), but instead at teaching them how to be syntacticians. What are the choices to be made? How can you tell what theory is best and why? These are the questions I'm struggling with in my own work, so it's been enlightening for me greatly as well. (Beyond this, I suppose my contribution just goes to make this course even more unique and weirder than the average university syntax class.)

In this spirit, I'm thinking that I'll teach the class my basic theory of VL grammar, then just give them a whole bunch of the more wacky and interesting sequences I've found and see what they can do with them. And, since I enjoyed it so much, I'm going to include Tim Godek's strip from today. Can you figure out what about it's structure makes me like it so much?

You're a good grammatical construction, Charlie Brown

I've recently been thinking quite a lot about how best to start testing my theories of visual language grammar. Since I'm in a psychology program, I've got to actually think of experiments that might yield reliable and significant results (hopefully).

One of the main ideas I had was starting off using a corpus of comic strips, so I wouldn't be biasing the study with my own drawings. I hit on the thought that Peanuts strips would be perfect for this since 1) there are a ton of wordless ones, 2) they're well recognized culturally, and 3) they use a fairly simple bare bones structure with 4) nearly always with 4 panels.

So, thanks to a very kind donation from Fantagraphics, I am now pouring through several volumes of The Complete Peanuts strips in search of all the wordless/minimal text ones I can find (there are a lot!). Hopefully, by summer I should be testing peoples intutions on the grammar of these strips, and eventually looking at their brainwaves while processing them (fun!).

One of the things that has jumped out at me is how so many of the strips use systematic patterns that I haven't noticed before. Previously, I've talked about the visual grammatical pattern of the 'Set up - Beat - Punchline' construction (as coined by Neal VonFlue). This is the pattern that sets up the joke with dialogue, then has a pause panel, then ends with the punchline. Well, Schulz seems to use a few other patterns a lot as well.

The most intruiging to me is one that is almost exactly like the SBP pattern, only the "beat/pause" panel isn't actually a pause: it's an "action" panel (SAP?). Instead of a passive type "rest," the space is filled by some wordless action that sets up the payoff with the final panel punchline. I've only looked at the oldest of the collections (the 1950s) and have only seen a few actual SBP constructions. I'm curious whether or not this SAP pattern preceded/led to the SBP one.

Another pattern has the first three panels as wordless depictions of an event, only to have a final panel with a punchline that explains or comments on the actions. This one happens extremely frequently, and sometimes takes on an additional characteristic of having the first panel depicting an action as well. It starts with an event that sets up the primary event that unfolds in the rest of the panels.

Patterns like this are fun to find, but can also be challenging theoretically. At least as far as developing a model for my visual grammar, sometimes I'm hesitant of how to notate certain panels, and often debate which is more correct. Imagine not only trying how best to describe how nouns and verbs combine, but also whether or not things are nouns and verbs in the first place and/or whether those categories are appropriate at all (when there's good evidence for all).

And, unlike with homework, there is often no answer key that I can check with someone else (except, hopefully, what my experiments will reveal). I've always found this "working without a net" to be a little scary, but at the same time exciting since it portends new and uncharted territory. I suppose it's the feeling of truly doing science instead of just learning it.


Note: As long as I'm giving thanks for donations, I should also mention the kind contributions of TopShelf, Drawn & Quarterly, Top Cow, Oni Press, and Dark Horse Comics. Their generosity will make a huge difference in these visual language studies and are greatly appreciated!! If you are from another company and would like to donate to this cause, please contact me...

When the going gets tough...

I am not too proud to say it: First-year statistics is kicking my butt. And, occasionally, driving me insane. Seriously. Sometimes I dream in excel spreadsheets.

So, for my spring break I've decided to commit myself to: the obvious solution I should have thought of long ago.

Housecleaning

What with my winter break coming to a close, I've been tending to some general housecleaning issues. I've recently updated my selection of "notable" blog posts as well as the ever useful Reference Bibliography. You'll also find (or perhaps have already found?) that I've filled in the tags for all my blog entries, which should help people find postings they're most interested in. Also, I'm debating pulling down some of my essays from the downloadable readings in anticipation of some upcoming publications, as well as general thoughts on the change and current status of my thinking. I'm not making any changes just yet, but if anything happens I'll make an annoucement here first probably.

Over break, I also discovered that amazon finally has a page selling my book Early Writings on Visual Language. If you buy from the seller with the name remarkably similar to mine, you'll essentially be going one extra middleman than buying from my site. However, if you already own the book, I encourage you to leave a review — negative, positive, or indifferent.

Finally, now that I'm back in Boston, regular updates of my current Meditations comic "Life is where Love is" is now back on schedule every Tues. and Thurs. ... at least until it ends. Then I'll see how i feel about future postings given my workload and energy this semester.

Speaking of which, it looks to be a good courseload coming up. I'm hoping to become more competant and comfortable with stats, am taking "Psychology of Language" and am my advisor's TA for "Syntax." Plus, I'll finally start getting to run my brainwave experiment! I'll also be speaking at a few conferences coming up. Definitely one in Boston, and maybe one in Connecticutt (still deciding on that). All in all, it looks to be a good semester...

I love this time of year

And now, for the 2006 edition of my annual Big Game poem:

Why so sad little stanfurd tree?
By Neil Cohn, November 2006

Oh why so sad little stanfurd tree?
Afraid you'll lose to Berkéley?

With a team like yours, I can guess the shame
That you must feel as you enter Big Game.

Your talent this year must be awfully thin,
Sitting at Pac-10's floor with a single win.

Poor stanfurd tree, you don't think its fair,
That you'll be pummeled by the mighty Golden Bear?

Perhaps you'll be lucky and you'll claim a score
But another loss to Cal will add to the previous four.

So, enjoy your one win and hope for a repeat
To avoid the embarrassment of an eleventh defeat.

But hope won't be enough come this Saturday
Cause at game's end, in Berkeley the Axe will stay!

Go Bears!

Neil's Research: Year One Projected

Over the past few weeks my advisors and I have been planning out what my first year project is going to be. We've decided it would be easier for me to jump into a straight-up psycholinguistics study than deal with designing a "comic" based study right away. So, my project is going to be looking at ERPs and semantic coercion.

What, you ask, are ERPs? ERP stands for "Event Related Potential" and its a measurement of the electrical activity of the human brain. How it works, is the experimenter puts a cap on people that measures their EEG, or electroencephalogram, which is the ongoing electrical activity in the brain. This electrical activity is all abuzz in your noggin' all the time. So, while doing certain tasks, this cap measures these brainwaves, which are then time-locked to those events that coincide with the tasks. The waveforms are then averaged out to reduce "noise," resulting (hopefully) in a waveform that can be informative about whatever task was performed.

Or, at least, that's as far as I know so far. I technically start learning how to run subjects in our lab on Monday.

And so, I'm sure you're wondering, what is "semantic coercion"? Glad you asked... Semantic Coercion is a linguistic phenomena that occurs when certain lexical items are paired up to create a meaning that is not explicit in the sentence. For example:

Tymmi began a comic.

In this sentence, you understand that the action is beginning to read or write a comic, yet nothing in the sentence is provided that tells you that. The information is illicited out of the combination of the words. This effect is not found if that information is provided:

Alexander finished his coffee (with coercion)
Alexander finished drinking his coffee (without coercsion)

So, my question for the next year will be... is anything special going on in the brain during sentences like these?

Which brings us to a final question you might ask... does this have anything to do with comics? And the answer to that, I'm afraid, lies in a future post sometime soon...

Webcomics writers unite!

So, I successfully returned from my trip to Boston, having found a place to live, which is good, because then I don't have to start school living out of a U-Haul. On this trip I was given an amazing welcome tour by Comixpedia's very own Kelly Cooper. First Alexander Danner, now Kelly... Boston is having a great showing by webcomics writers! It bodes well I think, since they can recruit subjects for my mad experiments for me.

Future Dr. Cohn

A few weeks ago I made a trip to Boston with the promise of some more details, so...

I am happy to announce that I will starting up grad school as a PhD candidate in Psychology at Tufts University in Boston starting in the Fall. I'm especially excited because my primary advisor will be Ray Jackendoff, whose work has always fascinated me, and who is what drew me to Tufts in the first place (my other advisors will be Phil Holcomb and Gina Kuperberg).

The program is in experimental cognitive psychology, meaning I will get to test my theories with lots of experiments on people. That means you can count on papers from me doing both behavioral testing and ERP brain studies about visual language in the coming years.

East Coast here I come!