Stephen C. Buckley, is the Associate Director of the Center for Digital Business and Center for Collective Intelligence at MIT.

I contacted the Center for Collective Intelligence at MIT since our two projects are quite similar, and had a chance to speak to Stephen Buckley, the Associate Director.
———————————————

How can people and computers be connected so that—collectively—they act more intelligently than any individuals, groups, or computers have ever done before? Why and under what circumstances do we need collective intelligence?

People right now don’t quite know what the secret sauce is for connecting people and computers in ways that at least seem to be intelligent.

Some people think that collective intelligence is some kind of magic pixie dust that you can sprinkle on top of any kind of a situation or problem, and it will automatically solve it. Then there are other people that criticize collective intelligence efforts, for example, like Wikipedia, because it’s not perfect, and therefore they believe that the only way to do things, organizationally, is through a centralized command and control structure.

Both schools of thought are probably equally wrong.

There are some instances, some problems that can be solved better collectively than they can, individually, and then there are those problems that lend themselves to a more traditional organizational structure.

Particularly in the sciences, the discourse that people have over the meanings of things, in the encyclopedia (Wikipedia), creates definitions that are more complete and also incorporate differences of opinion. There are some instances of entries in Wikipedia that aren’t perfect, and those tend to be the entries around which there are controversies. What this suggests is that Wikipedia is not a perfect tool for managing content.

We are really trying to find out a) what are the circumstances under which collective intelligence is a good idea, and b) what are the complex sets of incentives, motivations, and cultures that allow those efforts where you use collective intelligence, to be successful.

What has changed in the past few years that has given rise to Wikipedia, Digg.com, and so forth? Can we attribute this desire to harness collective intelligence to the availability of new technologies, or has there been an evolution in people’s social interactions that demands the use of collective intelligence?

We think that something fundamental has changed in the past couple of years, in the way that people are using computers.

We haven’t conducted any research that would allow us to unequivocally point out the fundamental change that is leading us towards more and more use of collective intelligence, but I can speculate.

Collective intelligence has existed probably since there were two or more people on the Earth. What I think has happened is 1) the network has grown in a way that a lot more people are connected to each other now, and 2) when you have more people connected to each other, you get a lot of people with very specialized skills, and also have access to information. And those two things combined, allow people with very narrow interests, i.e. if you are a particle physicists, interested in corks, you can literally join an online community with tens of thousands of other particle physicists who are interested in corks, and discussing even finer elements within that narrow field of interest.

So you can really bring a lot of intellectual force to bare, from all over the world, very quickly on a particular problem. And that just simply wasn’t very possible before.

And not only technology, but the evolution of social behavior also adds to this. People have come to realize that the problems we have are so complex, that it really is going to take a lot of people with specialized sets of skills to work on them.

Sometimes you can’t just understand something by just observing it from the outside. You have to create an instance of it that you can then study and watch the dynamics unfold, and see how people manage conflict, and also study the culture of the group, and ultimately, what people get out of it.

Why do people spend a fair portion of their time working on something that they don’t get any real compensation for. What kind of incentives can we provide to facilitate the process, or should the process be completely void of any reward system?

I am old enough to have lived through the whole knowledge management craze that went on in the 90’s, which centered around the idea that I will create a great big database, and you will come along and you will put everything you know into it. And then I can use the knowledge that you put into the database for some other purpose later on. So the idea was basically to get people to write everything down and then we will have it all.

As we know, it failed, because people didn’t want to spend the time to put stuff in the database when the incentive structure was that they were getting paid for billable hours and not for putting things in a database. Also, if they told you everything that they knew, they thought that this was essentially then reducing their own value.

Wikipedia is one of the famous successful examples of a company having figured out the complex set of incentives that get people to contribute. Ironically enough, the people I have talked to, who contribute to Wikipedia are the same people who grew up reading for example the encyclopedia Britannica.

An example of collective intelligence working, is Linux. Here you see people who in their spare time write code for this open source operating system, and check things, and fix bugs, and discuss feature sets and so on. And in the end, people who have never met each other, but have worked together for years, create this operating system.

The incentive structures are different here, because you run into a different kind of culture, the engineering culture. The rewards system in an engineering culture is elegance and functionality and so it’s a read ego boost for an engineer to create a piece of software that becomes the object of adulation for his fellow engineers.

All this is to say that we will probably find that in different types of situations there will be different kinds of cultures, and different sets of incentives that motivate those cultures to work collectively.

What ensures us that at the end, this will result in collective intelligence and not collective stupidity? While relying on collective intelligence, how do we ensure that we take into account a variety of different perspectives on any matter (from people of differing interest, backgrounds, areas of expertise)?

That is a big challenge to which there is no ‘silver bullet’ answer. Some of the ways in which others have dealt with it is to use some kind of reputation reporting. For example on eBay, people pride themselves on thousands of feedbacks and 99.9% positives. Or then there is Slashdot, where you can set your filters so you only see the users that post the most useful things on the discussion boards, and the ones below a certain threshold just get filtered out.

Or for example the rankings for book reviewers on Amazon. If you look at the top reviewers, I don’t even know where they get the time, they read like 10 books a week. And the write these very thoughtful and precise reviews that actually drive the sales of books because people read their reviews exclusively because they get to know the reviewers that they tend to agree with. So they read whatever the reviewer tells them to read.

One way to rate this ‘journalism’ would be on the basis of accuracy, usefulness, and I suppose there is another one, which is timeliness. Getting the scoop is a big consideration as well.

Ultimately, for any socially collaborative projects, there are some pitfalls that one must avoid. Could you elaborate on this, with special focus on the information cascade problem which seems to arise in nascent communities?

I have mixed feelings, I’ll confess, about online [open source] journalism, in some ways i like it because you are a lot more freer, and you can cover a lot more narrower areas, and in some ways, I guess one thing I don’t like about it is a kind of a philosophy that some people have that professional journalists are bad.

So if I were to give you some advice, I would say, don’t embrace that philosophy of some of the online-only publications. Instead, you should aspire to attain the professionalism of the most professional journalists, but do it in your own way.

Afterthought: This interview was followed by a breakdown of lessons learned that can be found here.
———

Muhameed Saleem is a Netscape Navigator and writes on his own blog The Mu Life where he studies the social bookmarking phenomenon.