Field of Science

Qing Wen!



Just one possible reaction if you use the wrong tone.





I've been an advocate for and member of Zipcar since my wife and I moved to Boston four and a half years ago. For that time period, I thought Zipcar was in every way superior to owning a car. Until last week, anyway. Now I'm reconsidering the car ownership issue.

Own or rent? That's the question.

It begins

I was already unhappy early in the week, having discovered Zipcar had overcharged us $375 over the past few months. At the beginning of August, I had added my wife as a driver on our account (we'd always had just one driver to save on the yearly membership fee) and also upgraded us to a fixed $75/month plan (which has some added benefits), having noticed that we'd spent more than $75 pretty much every month for the last year. I confirmed carefully with the representative that we would only be billed $75 combined, not $75 each.

It turns out, the hapless representative, rather than simply putting two of us on one account, made two accounts and put us each on both, and then charged us each every month. I didn't notice earlier as the charges appeared on different accounts, and I thought we only had one account, but the credit card charges were looking suspiciously large. It took a series of emails and a phone call to get that straightened out. They eventually agreed to refund us the bogus charges "as a one-time courtesy." That's a direct quote.


We had an overnight trip to our favorite New England B&B for Christmas weekend (seriously, this place is fantastic and has one of the best restaurants I've been to anywhere in the world).

Our room was even more charming than it looks. Yes, that's a working fireplace.

As usual, I booked a Zipcar for the purpose. I believe it was a Nissan Sentra, helpfully parked in our apartment building's garage. On Thursday, I got an email from Zipcar saying that due to an unforeseen circumstance, they were bumping us to a Civic in the Government Center garage.

The exterior of the building is distinctive.

I've rented cars once or twice from that location. During the day, it's ok. At night, it's spooky as hell. I'd say it's deserted, but there are occasionally roving bans of teens doing who knows what.

I couldn't find a good picture of the interior.
This illustration is true to the spirit of the place.

So I emailed Zipcar, explaining that I didn't really like returning cars to that location at night, so was there maybe another car nearby I could use. Or could the Civic be relocated somewhere more pleasant for a couple days?


I don't know if anybody read that email, since I never got a reply. I did get another email on Friday, though, saying that due to another unforeseen circumstance, my reservation had been moved to a Smart Car parked in Somerville (a 15-20 minute drive from where we live).

Did I mention that the B&B was a 3 hour drive away? I didn't really want to drive a Smart Car on the highway for 3 hours, and I wasn't sure our stuff would fit (I had planned on bringing skiis). So I emailed, saying if I it was a choice between a Civic parked in the Dungeon of Dispair or a Smart Car, I'd take my chances with the dungeon.

This time, I got a quick email saying there were no other cars available. So I called and explained my situation. A very polite representative explained that there really were no other cars nearby, but would I take a Mazda 3 in Arlington (3 suburbs out from Boston, where I live)? They'd pick up the cab fare. Either that, or I could have $200 towards some other form of transportation. Or I could cancel my reservation. $200 wasn't going to cover a standard car rental (I checked), and the B&B reservation was nonrefundable (plus we'd been looking forward to it), so we went with the car in Arlington.


We took a cab out to Arlington in the morning. It took maybe 20 minutes (we helpfully live next to I-93, making getting out of the city easy -- Thank you, Tip O'Neil and Ted Kennedy) and cost $33.75.

We'd only just gotten back onto I-93 in the direction of Vermont when there was a loud pop under the car and it sounded like something was dragging. On inspection (I took the next exit and pulled over), there was some piece of plastic hanging loose. The plastic itself didn't look problematic, but I wasn't sure what it had previously been holding in place. So I called Zipcar.

The representative agreed that the car was not safe to drive and asked us to return the car to its original location, and could we perhaps take a Prius from Wellesley College instead? If we needed a cab ride, they'd cover the fare. I pointed out that (a) we were already running pretty late, and (b) a cab fare to Wellesley was going to be pretty serious, esp. on top of our cab fare to Arlington, so could I just drive the car to Wellesley, drop it off there and take the Prius. She said that wasn't possible, since when the mechanics came out to service the Mazda, they wouldn't know where it was. I said if that was the problem, I was happy to tell the mechanics that the car was in Wellesley.

She put me on hold.

After a brief wait, she came back on the line to apologize, saying she hadn't gotten permission to drop the car off in Wellesley. Did I want to make the switch anyway? Or there was another car in Salem, MA, if we wanted.

Witch trials: Popular entertainment in Salem, MA

I've actually wanted to see Wellesley for a while (I like college towns), so we went with Wellesley. We called yet another cab (have I mentioned this was Christmas? Not a lot of cabs wanting to go to a deserted college town) and went to Wellesley. That fare was $66.05, including a Christmas-appropriate tip.

Finally, we got the Prius and set off. We had been traveling for 2 1/2 hours and were now farther from our destination than when we started.

View Larger Map

Our itinerarary: A: Home, B: Mazda in Arlington, C: Roughly where the car broke, D: Back in Arlington, E: Wellesley College. Vermont is in the North.


The rest of Saturday was pretty good, and the B&B was everything we remembered (dinner, which was, as always, excellent, included what may be the perfect bread, from Orchard Hill Breadworks). Sunday morning we heard rumors that a serious blizzard was heading our way, though it wasn't expected to be bad until evening (before we had left, the weather report had put the chance of snow at only 30%). We got a slightly earlier start than we had planned, stopped at a few places on the way back.

Somewhere around 3:30 or 4:00, we entered Massachusetts and it began to snow. The state had put up blizzard warnings on the roads, requesting everyone to get off the road and go home. If we had been going straight home -- as we would have were we driving that much-mourned Sentra -- that wouldn't have been a problem. But we were going to Wellesley.

At least, we tried. As we neared Wellesley, the snow got very bad, and I frankly wasn't that comfortable driving, particularly once we left the highway and the streets weren't as well-plowed. My wife called a cab company to make sure we could get a ride back to Boston. They agreed to take us, but then called back shortly thereafter, saying that nobody was driving anywhere, didn't we know there was a blizzard going on? I think they made the right decision.

I did know there was a commuter rail station in Wellesley. We called the MBTA to see if the trains were still running. They said the trains would most likely run, but with significant delays. The next one wouldn't be for 3 hours. Oh, and the station we'd be waiting at is outside. In a blizzard.

As a backup plan, we checked to see if there was anywhere we could stay in the night in Wellesly. However, as Wellesley doesn't have any hotels, there appeared to be only one option:

The only available room in Wellesley last night.

We called up Zipcar to consult. They agreed to let us leave the car in Boston at no charge, as long as we told them where the car was. I'll give them credit for that decision, at least. We drove home, very slowly.


I wish I could say the story was over, but this morning I checked my email and saw that we were billed, not only for the Prius, but also for the Mazda 3 (the one that broke down). Plus, there was a late fee for returning the Mazda late. It seems that when the representative switched our rental from the Mazda to the Prius, she did so before we actually got back to Arlington. I sent in another email this morning. We can hope that they'll remove those charges "as a one-time courtesy."

I realize that owning a car has its own hassles. I don't expect Zipcar to be perfect, either. Everyone's allowed a bad week. As long as it's just one week. And as long as they give me my money back.

What did I learn from this experience? What I learned -- and what you should take home from this as well -- is to get your bread from Orchard Hill. Because it is fucking awesome bread.

*Update: Tuesday*

This morning I got a call from someone higher up in customer service at Zipcar, who listened to the whole story. She took the numbers for the cab rides in order to reimburse us directly, rather than my having to send in the receipts, which was nice. She also comped the entire weekend trip and added a $50 driving credit, which I also appreciate. The part I cared about more was that she at least seemed very interested in improving the service such that such problems would not be repeated or would be mitigated more quickly when they do. If this reflects a real commitment to efficient service, then hopefully this last week is an aberration, and we'll be able to go back to trusting and relying on Zipcar, as we have in the past.

*Another Update: Tuesday*

Now the vice president for member services has called to apologize in addition. It's great that they take this stuff seriously. I was going to take a temporary break from Zipcar and use a regular rental car company for some upcoming stuff, but now I think I'll give them another shot.

Crowdsourcing My Data Analysis

I just finished collecting data for a study. Do you want to help analyze it?


What makes a pun funny? If you said "nothing," then you should probably skip this post. But even admirers of puns recognize that while some are sublime, others are ... well, not.

Over the last year, I've been asking people to rate funniness of just over 2300 different puns. (Where did I get 2300 puns? The user-submitted site PunoftheDay. PunoftheDay also has funniness ratings, but I wanted a bit more control over how the puns were rated and who rating them.).

Why care what makes puns funny?

There are three reasons I ran this experiment. I do mostly basic research, and while I believe in its importance and think it's fun, the idea of doing a project I could actually explain to relatives was appealing. I was partly inspired by Zenzi Griffin's 2009 CUNY talk reporting a study she ran on why parents call their kids by the wrong names (typically, calling younger children by elder children's names), work which has now been published in a book chapter.

Plus, I was just interested. I mean: puns!

Finally, I was beginning a line of work on the interpretation of homophones. One of the best-established facts about homophones is that we very rapidly suppress context-irrelevant meanings of words -- in fact, so rapidly that we rarely even notice. If your friend said, "I'm out of money, so I'm going to stop by the bank," would you really even notice considering that bank might mean the side of a river?

A river bank. 
photo: Istvan, creative commons 

A successful pun, on the other hand, requires that at least two meanings be accessed and remain active. In some sense, a pun is homophone processing gone bad. By better understanding puns, I thought I might get some insight into language processing.


As already mentioned, my first step down this road was to collect funniness ratings for a whole bunch of puns. I popped them into a Flash survey, called it Puntastic, and put it on the Games With Words website. The idea was to mine the data and try to find patterns which could then be systematically manipulated in subsequent experiments.

It turns out that there are a lot of ways that 2300 puns can be measured and categorized. So while I have a few ideas I want to try out, no doubt many of the best ones have not occurred to me. Data collection was crowdsourced, and I see no reason why the analyses shouldn't be as well.

I have posted the data on my website. If you have some ideas about what might make one pun funnier than another -- or just want to play around with the data -- you are welcome to it. Please post your findings here.

If you are a researcher and might use the data in an official publication, please contact me directly before beginning analysis (gameswithwords$at* just so there aren't misunderstandings down the line. Failure to get permission to publish analyses of these data may be punished by extremely bad karma and/or nasty looks cast your way at conferences.

The results so far...

Unfortunately for the crowd, I've already done the easiest analyses. The following are based on nearly 800 participants over the age of 13 who listed English as both their native and primary languages (there weren't enough non-native English speakers to conduct meaningful analyses on their responses).

The average was 2.6 stars out of 7 (participants could choose anywhere from 1 to 7 stars, as well as "I don't get it," which was scored as -1 for these analyses), which says something either about the puns I used or the people who rated them.

First I looked at differences between participants to see if I could find types of people who like puns more than others. There was no significant difference in overall ratings by men or women.

I also asked participants if they thought they had good or poor social skills. There was no significant difference there, either.

I also asked them in they had difficulty reading or if they had ever been diagnosed with any psychiatric illnesses, but neither of those factors had any significant effect either (got tired of making graphs, so just trust me on this one).

The effect of age was unclear.

It was the case that the youngest participants produced lower ratings than the older participants (p=.0029), which was significant even after a conservative Bonferroni correction for 15 possible pairwise comparisons (alpha=.0033). However, the 10-19 year-olds' ratings were also significantly lower than the 20-29 year-olds' (p=.0014) and the 30-39 year-olds' (p=.0008), but obviously this was not true of the 40-49 year-olds' or 50-59 year-olds' ratings. So it's not clear what to make of that. Given that the overall effect size was small and that this is an exploratory analysis, I wouldn't make much of the effect without corroboration from an independent data set.

The funniest puns

The only factor I've looked at so far that might explain pun funniness is the length of the joke. I considered only the 2238 puns for which I had at least 5 ratings (which was most of them). I asked whether there might be a relationship between the length of the pun and how funny it was. I could imagine this going either way, with concise jokes being favored (short and sweet) or long jokes having a better lead-up (the shaggy dog effect). In fact, the correlation between pun ratings and length in terms of number of characters (r=.05) or in terms of number of words (r=.05) were both so small I didn't bother to do significance tests.

I broke up the puns into five groups according to length to see if maybe there was a bimodal effect (shortest and longest jokes are funniest) or a Goldilocks effect (average-length jokes are best). There wasn't.

In short, I can't tell you anything about what makes some people like puns more than others, or why people like some puns more than others. What I can tell you is which puns people did or didn't like. Here are the top 5 and bottom 5 puns:

1. He didn't tell his mother that he ate some glue. His lips were sealed.
2. Cartoonist found dead in home. Details are sketchy.
4. The frustrated cannibal threw up his hands.
5. Can Napoleon return to his place of birth? Of Corsican.
2234. The Egyptian cinema usherette sold religious icons in the daytime. Sometimes she got confused and called out, 'Get your choc isis here!'
2235. Polly the senator's parrot swallowed a watch.
2236. Two pilgrims were left behind after their diagnostic test came back positive.
2237. In a baseball season, a pitcher is worth a thousands blurs.
2238. He said, "Hones', that is the truth', but I knew elide.

Ten points to anyone who can even figure out what those five puns are about. Mostly participants rated this as "I don't get it."

BTW Please don't take from this discussion that there hasn't been any serious studies of puns. There have been a number, going back at least as far as Sapir of the Sapir-Whorf hypothesis, who wrote a paper on "Two Navaho Puns." There is a well-known linguistics paper by Zwicky & Zwicky and at least one computer model that generates its own puns. However, I know a lot less about this literature than I would like to, so if there are any experts in the audience, please feel free to send me links.

Paper submitted




Here's the first draft, according to Wordle:

Here's draft 2:


I guess we'll see when the reviews come back in.

Do you speak Japanese?

Do you speak Japanese?



Huh? Pronoun processing?




More posts on pronouns: here, here and here.

missing 2

One of the formulas in the last post was missing a 2. Everything has now been recalculated. Some numbers changed. The basic result is that some of the numbers are not quite as dire as I had stated: the original example experiment, which had 15 participants per condition and an effect significant at p=.05 has a 51% chance of replicating (in the sense of producing another significant p-value when re-run exactly), again assuming the effect was real and the effect size is as measured in the first experiment.

When should an effect be called significant?

note: This post originally contained an error in the mathematics, which Tal of Citation Needed kindly pointed out. This error has been corrected.

In the thread following my earlier post on false positives, Tal made the observation that in a typical study that is significant at the p=.05 level has a 50% chance of being replicated.

It turns out that this depends heavily on what you mean by replicate. I'm going to work through some numbers below. Again, stats isn't my specialty, so please anyone jump in to correct errors. But I think I've got the general shape of the issues correct.

I got a significant result! Can I get it again?

Let's say you ran an experiment comparing the IQ scores of 15 people who prefer kiwi with the IQ scores of 15 people who prefer rambutan. You find that people who prefer rambutan have IQs 11.2 points higher than those who prefer kiwi. Assuming the standard deviations is 15 (which is how IQ tests are normalized), then that should give you a t-value of 11.2 / (15 * (2/15)^.5) = 2.04 and a p-value of about .05.

So you've now got a significant result! You tell all your friends, and they try running the same experiment. What are the chances they'll get the same results, significant at the p=.05 level?

The chances are not great. Even assuming that the underlying effect is real (rambutan-eaters really are smarter), your friends will only replicate your result about 51%, assuming they use exactly the same methods (according to a nifty power calculator found online here).

Define "get it"

Of course, we were assuming above that rambutan-eaters really are 11.2 IQ points smarter than kiwi-eaters (BTW I like both kiwi and rambutan, so nothing is meant by this example). In which case, your friends might not have gotten results significant at the p=.05 level, but they very likely found higher average IQs for their samples of rambutan-eaters relative to kiwi-eaters. And of course, what we really care about is how easy it will be to replicate the rambutan/kiwi difference, not how easy it will be to get the significant p-value again. The point of science is not to be able to predict statistically-significant differences but simply to predict differences. It's well beyond my statistical abilities to say how often this would happen, but hopefully someone will step up in the comments and let us know.

In practice, though, other people are only going to follow up on your effect if they can replicate it at the standard p=.05 level. What can we do to improve the chances of replicability?

Lower alphas

Let's suppose your effect had been significant at the p=.01 level. We can manage that while keeping the effect-size the same (11.2 IQ points) if we increase our sample to 26 kiwi-eaters and 26 rambutan-eaters (t = 8/(15 * (2/26)^.5) = 2.7). Now our chance of getting another significant result at the p=.01 level is ... 52%.

But we don't really care about getting a p=.01 again; we want to get the result again at the p=.05 level, which should happen around 76% of the time. Now, what if we had a result significant at the p=.001 level the first time around? We'd have needed about 42 subjects per condition. The chance or replicating that at the p=.05 level is 92%.

p-value       #subjects/condition           Likelihood of repeating at p=.05 level
.05                         15                                 51%
.01                         26                                 76%
.001                       42                                 92%


Of course, there are things that I'm not considering here, like the possibility that your original experiment underestimated the effect size. For instance, let's say that the true effect size is 15 IQ points (which is a lot!). Now, your chances of finding an effect significant at the p=.05 level with only 15 participants per condition is 75%. That's a lot better than what we started with, though not perfect. To have an effect large enough to see it 95% of the time at the p=.05 level, it would need to be over 20 IQ points, which is a monstrously large effect. Incidentally, if you ran this study with 15 rambutan-eaters and 15 kiwi-eaters and found a 20 IQ point effect, that would be significant below the p=.001 level.

What I get from all this is that if you want a result that you and others will be able to replicate, you're going want the p-value in your original experiment to have been lower than p.05.

Poll: Do You Care about Effect Size?






Why I rarely care about effect size








Slate calls for More Republican Scientists

Daniel Sarewitz of Slate worries that there aren't enough Republican scientists. Is it any wonder that Republicans don't trust science, if it's all coming from the laboratories of Democrats?

I don't know. Is it?

What would a Republican scientist look like? Would she accept the reality of man-made global warming? Evolution? Would she be aware that gay couples are at least as good parents as straight couples? Or that states that allow gay marriage and civil unions have lower divorce rates than those that don't. That true Keynesian economic stimulus works pretty well, and that tax cuts for the rich have little effect on the economy? That the health care system in Western Europe is cheaper and more effective than the American health care system?

As Colbert once noted, reality has a well-known liberal bias.

I strongly believe that communities are stronger when they are made up of people with diverse viewpoints. There is no benefit to a community made up of people with diverse facts.

Yes, we should be worried by the paucity of Republican scientists. But what does Sarewitz mean by saying we need to make the scientific community "more welcoming" to Republicans? If that means wearing elephant pins, I'll go along with it. If it means abandoning facts for fiction...

And for my next trick, I'll make this effect disappear!



Lehrer's focus in this article is shrinking effects.

No, not this one.

Some experimental effects seem to shrink steadily over time:


Where I get off the Bus

So far so good, except here's Lehrer again:


The Magic of Point O Five


and again:



Messing with that Point O Five




a normal distribution





Run enough experiments and...



Next I try oranges....



Data peaking




Data massaging




Better estimates






No better prediction of replication than replication




So why not replicate more?






That's my conclusion. Here's Lehrer's:
I say it again: huh?

Priest, Altars and Peer Review

David Dobbs at Neuron Culture is complaining about NASA and peer review:
A NASA spokesperson has dismissed a major critique of the Science arsenic bug paper based not on the criticism's merits, but on its venue -- it appeared in a blog rather than a peer-reviewed journal. Apparently ideas are valid (or not) based on their content, or even the reputation of the author, but on where they're published.
I'm not known for my strong endorsement of the fetishism of peer review, but even so I think Dobbs is being somewhat unfair. My reading of history is that scientists have been plugging the peer-review mantra because they're tired of having to respond to ignorant assholes who appear on Oprah spouting nonsense. I mean, yes, you can address wacko claims about vaccines causing autism or the lack of global warming on their merits (they have none), but it gets tiresome to repeat. In any case, relatively few members of the public can follow the actual arguments, so it becomes an issue of who you believe. And that's a hard game to win, since saying "so-and-so doesn't know what they're talking about" sounds elitist even when it's true, and "elitism" (read: "meritocracy") is for some reason unpopular.

Focusing on peer review as a mechanism for establishing authority is convenient, because the public (thinks it) understands the mechanisms. You're not saying, "Believe me because I am a wise scientist," but "Believe the documented record." And since Jenny McCarthy doesn't publish in peer-reviewed journals, you can (try to) exclude her and other nuttos from the conversation.

So I think there are good reasons for a NASA spokesman, when speaking with a reporter, to dismiss blogs. Taking a critique in a blog seriously in public is only going to open the floodgates. I mean, there are a *lot* of blogs out there. That doesn't mean that the scientists involved aren't taking the a series critique by a serious scientist seriously just because the criticism appeared in a blog. I hope that they are, and we don't want to read too much into NASA's official statement.

All that said, I'm not sure focusing on peer-reviewed science has been helping very much. I mean, McCarthy still gets booked on Oprah anyway.

More Politics

As expected, the President appears to be caving on massive tax-breaks for the mega-wealthy while cutting back on services for everyone else and key investments for the future (in the name of fiscal responsibility). The fact that this is a Republican proposal doesn't make him any less responsible. I'll be voting for someone else in two years.

Google Translate Fail

Google Translate's blog:
There are some things we still can't translate. A baby babbling, for example. For the week of November 15th we are releasing five videos of things Google can’t translate (at least not yet)! Check out the videos and share them with your friends. If you can think of other things you wish Google translated (like your calculus homework or your pet hamster), tweet them with the tag #GoogleTranslate. We’ll be making a video of at least one of the suggestions and adding it to our page.
What do I wish Google Translate could translate? I'll bite. How about Russian? Or Japanese?

I mean, have the folks over at GT ever actually used their product? It's not very good. I'll admit that machine translation has improved a lot in recent years, but I doubt it's as good as a second-year Spanish student armed with a pocket dictionary.

Nothing against the fine engineers working at Google. GT is an achievement to be proud of, but when they go around claiming to have solved machine translation, it makes those of us still working on the problems of language look bad. It's hard enough to convince my parents that I'm doing something of value without Google claiming to have already solved all the problems.