Tales from the Loonybin

I had the chance to have lunch today with Luis von Ahn. He’s a fascinating guy and does some really cool work on Human Computation. What is human computation? It’s the process of using the computer to have people solve problems that people are good at, but computers aren’t. For example, people are really good at looking at an image and seeing what’s in it, but computers are pretty lousy at it.

Human computation uses people to do all of the work of labeling the images. How do you get people to actually do the work? By turning it into a game. The ESP Game pairs random players on the internet. Each player has to guess the same word as the other player. The only thing they have in common is that they see the same image.

Give it a try — it’s a lot of fun (and addictive!). The best part is, even though you’re “wasting” time, you’re doing something productive!

What other things could we turn into a game? Could I get a bunch of random people on the Internet to write my dissertation?

I just spent longer than I would like to admit trying to track down why my code was barfing on unicode characters (like the é in café). It turns out that Python 2.3.5 (the version shipped with Mac OS X Tiger) converts HTML charrefs (e.g. &eaigu; — é) to str characters rather than to unicode characters. Apparently, sgmllib (which is used by htmllib) uses chr() instead of unichr().

For various reasons, I need to be able to take HTML formatted text and convert it to plain text. The python standard library includes htmllib, which makes that a fairly easy task. I just had to modify it slightly to get it handle Unicode. Here’s my solution.

Update: Added fix for entityrefs, too (e.g.  &#233)

__(’Read the rest of this entry »’)