Treading on the Commons: Book Recommendations and the Wisdom of the Crowd

I love the idea of recommendation services like LibraryThing, Bibliophil and Books iRead. The main reason I like these services is because of their potential to identify items for me that I may never encounter on my own. You have to accept that I am a) a busy parent with little time to read, b) a busy parent with even less time to find a book to read, c) a librarian who spends too much time on his computer and d) someone who likes pleasant surprises and who has tolerance for entropy.

As I add books to these recommendation services, I am becoming increasingly aware of a problem, similar to the ever-present “Tragedy of the Commons.” The problem is this: with all the classics in my collection I have a hard time getting useful recommendations.

I like classics. As someone who enjoys classics, I put some of my favorites into the database. Things like William Faulkner’s The Sound and the Fury. I have it in my personal collection, largely because it is the sort of thing that my kids might enjoy reading when they are older. If I’m going to do ‘trash’ reading, or popular reading — I’m not going to buy, but instead I’ll use the library. I enjoyed A Spot of Bother, for instance, but I have no desire to read it twice. That’s why borrow-and-return works for me.

In the world of book recommending algorithms, this is a problem. It came to a head today when I searched for a recommendation using Books iRead and of the 10 recommendations I trudged through, 8 were in the “obvious classic genre.” 3 were Shakespeare. 1984 showed up, as did To Kill a Mockingbird, Oliver Twist, more than one Harry Potter, The Grapes of Wrath, Alice in Wonderland, The Lord of the Rings and so on. I could have asked any English-speaking (and maybe a few non-English speaking too) 17-year-old on earth to come up with these books as recommendations.

But that’s not the real problem. If I want to keep 1984 from showing up as a recommendation again, I need to add it to my collection as a “Read It, Reading It or Want to Read It.” But if I add it to my collection, it’s all the more likely that some other person is going to have to suffer the same ordeal. So I’m stuck in a conundrum: do I add it and save myself some misery, or do I ignore it and continue to grumble everytime I see it on my list.

(Remember this: there’s always a third option. I removed the application from my [rarely used now] Facebook account and went elsewhere).

Now, I am picking on Books iRead, but the other applications have similar problems. At the heart of the matter is the nature of social information in the first place. Our world is full of influences — both traditional and commercial — that hit on our collective ability to coordinate our interests. When I want to know how I am similar to the crowd, simple algorithms work fine. If I want to know how I’m different, there’s a problem.

The Solution:

The solution is empathy, understanding, broad-thinking, letting people help computers think rather than the other way around. That is why I am going to put in some love for LibraryThing over its competitors.

Library Thing is not about its algorithms and that’s the difference. You can tell by its attempts at making the website human. Yes, their “people with your books also have. . .” search gives me the usual suspects like Henry James, and D.H. Lawrence — but then I can just switch to a tags or “special sauce” recommendation. There’s also a neat “easy linking” tool and api, that will try to guess titles from keywords in a URL. For instance, here is the result for “ panky/“. Currently I am using this service as part of a Library Thing/Twitter mashup. (It’s vaporware right now, except to people who have followed me on Twitter). There are more surprises coming that direction as well.

But the real secret of Library Thing’s success has little to do with the range of services it offers, but instead in Tim Spalding’s understanding of what libraries are and how they work. For one, he tapped the quagmire that is Z39.50 and took his service one step beyond what just a re-hash of what Amazon has to offer. He added a “talk” section to his website, because he understands that books are one way we connect with other human beings. And he hires librarians too.

So, there’s no wonder Tim Spalding is in Library Journal’s Movers and Shakers. But forget that, give him the freakin’ Reader’s Nobel Peace Prize. Or better yet, howzabout an “I Love You” subscription to the Library Thing service. He deserves to be sitting on a pot of gold, and you’ll be sitting on the (ahem) pot with lots of good reading. Maybe we could also get him a better set of data than that Z39.50 stuff we’ve been handing him too, eh?

And you know what? I also think the same kind of understanding is what makes MetaFilter successful as well. Jessamyn West is revered as a near goddess there, and I can see why. There’s a good mix of the social and the authoritative — which is what librarians have been all about for, like, 100s of years.

In the end, I think librarians rock. The main problems we have occur when we get in our own way — as in insisting on complicated standards where more simple and flexible standards will do. Viva Libraria!