Clay Shirky on Cognitive Surplus

Clay Shirky (@cshirky) has noticed the trend of talented people putting five or six hours an evening to work instead of to waste. Add that up across a million or ten million people and the output is astonishing. He calls it cognitive surplus and it's one of the underappreciated world-changing stories of our time.



Clay’s latest book is “Here Comes Everybody”, about organizing without organizations.

Transcription

I was recently reminded of some reading I did in college, way back in the last century, by a British historian arguing that the critical technology, for the early phase of the industrial revolution, was gin.

The transformation from rural to urban life was so sudden, and so wrenching, that the only thing society could do to manage was to drink itself into a stupor for a generation. The stories from that era are amazing-- there were gin pushcarts working their way through the streets of London.

And it wasn't until society woke up from that collective bender that we actually started to get the institutional structures that we associate with the industrial revolution today. Things like public libraries and museums, increasingly broad education for children, elected leaders--a lot of things we like--didn't happen until having all of those people together stopped seeming like a crisis and started seeming like an asset.

It wasn't until people started thinking of this as a vast civic surplus, one they could design for rather than just dissipate, that we started to get what we think of now as an industrial society.


If I had to pick the critical technology for the 20th century, the bit of social lubricant without which the wheels would've come off the whole enterprise, I'd say it was the sitcom. Starting with the Second World War a whole series of things happened--rising GDP per capita, rising educational attainment, rising life expectancy and, critically, a rising number of people who were working five-day work weeks. For the first time, society forced onto an enormous number of its citizens the requirement to manage something they had never had to manage before--free time.

And what did we do with that free time? Well, mostly we spent it watching TV.

We did that for decades. We watched I Love Lucy. We watched Gilligan's Island. We watch Malcolm in the Middle. We watch Desperate Housewives. Desperate Housewives essentially functioned as a kind of cognitive heat sink, dissipating thinking that might otherwise have built up and caused society to overheat.

And it's only now, as we're waking up from that collective bender, that we're starting to see the cognitive surplus as an asset rather than as a crisis. We're seeing things being designed to take advantage of that surplus, to deploy it in ways more engaging than just having a TV in everybody's basement.

This hit me in a conversation I had about two months ago. As Jen said in the introduction, I've finished a book called Here Comes Everybody, which has recently come out, and this recognition came out of a conversation I had about the book. I was being interviewed by a TV producer to see whether I should be on their show, and she asked me, "What are you seeing out there that's interesting?"

I started telling her about the Wikipedia article on Pluto. You may remember that Pluto got kicked out of the planet club a couple of years ago, so all of a sudden there was all of this activity on Wikipedia. The talk pages light up, people are editing the article like mad, and the whole community is in an ruckus--"How should we characterize this change in Pluto's status?" And a little bit at a time they move the article--fighting offstage all the while--from, "Pluto is the ninth planet," to "Pluto is an odd-shaped rock with an odd-shaped orbit at the edge of the solar system."

So I tell her all this stuff, and I think, "Okay, we're going to have a conversation about authority or social construction or whatever." That wasn't her question. She heard this story and she shook her head and said, "Where do people find the time?" That was her question. And I just kind of snapped. And I said, "No one who works in TV gets to ask that question. You know where the time comes from. It comes from the cognitive surplus you've been masking for 50 years."

So how big is that surplus? So if you take Wikipedia as a kind of unit, all of Wikipedia, the whole project--every page, every edit, every talk page, every line of code, in every language that Wikipedia exists in--that represents something like the cumulation of 100 million hours of human thought. I worked this out with Martin Wattenberg at IBM; it's a back-of-the-envelope calculation, but it's the right order of magnitude, about 100 million hours of thought.

And television watching? Two hundred billion hours, in the U.S. alone, every year. Put another way, now that we have a unit, that's 2,000 Wikipedia projects a year spent watching television. Or put still another way, in the U.S., we spend 100 million hours every weekend, just watching the ads. This is a pretty big surplus. People asking, "Where do they find the time?" when they're looking at things like Wikipedia don't understand how tiny that entire project is, as a carve-out of this asset that's finally being dragged into what Tim calls an architecture of participation.

Now, the interesting thing about a surplus like that is that society doesn't know what to do with it at first--hence the gin, hence the sitcoms. Because if people knew what to do with a surplus with reference to the existing social institutions, then it wouldn't be a surplus, would it? It's precisely when no one has any idea how to deploy something that people have to start experimenting with it, in order for the surplus to get integrated, and the course of that integration can transform society.

The early phase for taking advantage of this cognitive surplus, the phase I think we're still in, is all special cases. The physics of participation is much more like the physics of weather than it is like the physics of gravity. We know all the forces that combine to make these kinds of things work: there's an interesting community over here, there's an interesting sharing model over there, those people are collaborating on open source software. But despite knowing the inputs, we can't predict the outputs yet because there's so much complexity.

The way you explore complex ecosystems is you just try lots and lots and lots of things, and you hope that everybody who fails fails informatively so that you can at least find a skull on a pikestaff near where you're going. That's the phase we're in now.

Just to pick one example, one I'm in love with, but it's tiny. A couple of weeks one of my students at ITP forwarded me a a project started by a professor in Brazil, in Fortaleza, named Vasco Furtado. It's a Wiki Map for crime in Brazil. If there's an assault, if there's a burglary, if there's a mugging, a robbery, a rape, a murder, you can go and put a push-pin on a Google Map, and you can characterize the assault, and you start to see a map of where these crimes are occurring.

Now, this already exists as tacit information. Anybody who knows a town has some sense of, "Don't go there. That street corner is dangerous. Don't go in this neighborhood. Be careful there after dark." But it's something society knows without society really knowing it, which is to say there's no public source where you can take advantage of it. And the cops, if they have that information, they're certainly not sharing. In fact, one of the things Furtado says in starting the Wiki crime map was, "This information may or may not exist some place in society, but it's actually easier for me to try to rebuild it from scratch than to try and get it from the authorities who might have it now."

Maybe this will succeed or maybe it will fail. The normal case of social software is still failure; most of these experiments don't pan out. But the ones that do are quite incredible, and I hope that this one succeeds, obviously. But even if it doesn't, it's illustrated the point already, which is that someone working alone, with really cheap tools, has a reasonable hope of carving out enough of the cognitive surplus, enough of the desire to participate, enough of the collective goodwill of the citizens, to create a resource you couldn't have imagined existing even five years ago.

So that's the answer to the question, "Where do they find the time?" Or, rather, that's the numerical answer. But beneath that question was another thought, this one not a question but an observation. In this same conversation with the TV producer I was talking about World of Warcraft guilds, and as I was talking, I could sort of see what she was thinking: "Losers. Grown men sitting in their basement pretending to be elves."

At least they're doing something.

Did you ever see that episode of Gilligan's Island where they almost get off the island and then Gilligan messes up and then they don't? I saw that one. I saw that one a lot when I was growing up. And every half-hour that I watched that was a half an hour I wasn't posting at my blog or editing Wikipedia or contributing to a mailing list. Now I had an ironclad excuse for not doing those things, which is none of those things existed then. I was forced into the channel of media the way it was because it was the only option. Now it's not, and that's the big surprise. However lousy it is to sit in your basement and pretend to be an elf, I can tell you from personal experience it's worse to sit in your basement and try to figure if Ginger or Mary Ann is cuter.

And I'm willing to raise that to a general principle. It's better to do something than to do nothing. Even lolcats, even cute pictures of kittens made even cuter with the addition of cute captions, hold out an invitation to participation. When you see a lolcat, one of the things it says to the viewer is, "If you have some sans-serif fonts on your computer, you can play this game, too." And that's message--I can do that, too--is a big change.

This is something that people in the media world don't understand. Media in the 20th century was run as a single race--consumption. How much can we produce? How much can you consume? Can we produce more and you'll consume more? And the answer to that question has generally been yes. But media is actually a triathlon, it 's three different events. People like to consume, but they also like to produce, and they like to share.

And what's astonished people who were committed to the structure of the previous society, prior to trying to take this surplus and do something interesting, is that they're discovering that when you offer people the opportunity to produce and to share, they'll take you up on that offer. It doesn't mean that we'll never sit around mindlessly watching Scrubs on the couch. It just means we'll do it less.

And this is the other thing about the size of the cognitive surplus we're talking about. It's so large that even a small change could have huge ramifications. Let's say that everything stays 99 percent the same, that people watch 99 percent as much television as they used to, but 1 percent of that is carved out for producing and for sharing. The Internet-connected population watches roughly a trillion hours of TV a year. That's about five times the size of the annual U.S. consumption. One per cent of that is 100 Wikipedia projects per year worth of participation.

I think that's going to be a big deal. Don't you?

Well, the TV producer did not think this was going to be a big deal; she was not digging this line of thought. And her final question to me was essentially, "Isn't this all just a fad?" You know, sort of the flagpole-sitting of the early early 21st century? It's fun to go out and produce and share a little bit, but then people are going to eventually realize, "This isn't as good as doing what I was doing before," and settle down. And I made a spirited argument that no, this wasn't the case, that this was in fact a big one-time shift, more analogous to the industrial revolution than to flagpole-sitting.

I was arguing that this isn't the sort of thing society grows out of. It's the sort of thing that society grows into. But I'm not sure she believed me, in part because she didn't want to believe me, but also in part because I didn't have the right story yet. And now I do.

I was having dinner with a group of friends about a month ago, and one of them was talking about sitting with his four-year-old daughter watching a DVD. And in the middle of the movie, apropos nothing, she jumps up off the couch and runs around behind the screen. That seems like a cute moment. Maybe she's going back there to see if Dora is really back there or whatever. But that wasn't what she was doing. She started rooting around in the cables. And her dad said, "What you doing?" And she stuck her head out from behind the screen and said, "Looking for the mouse."

Here's something four-year-olds know: A screen that ships without a mouse ships broken. Here's something four-year-olds know: Media that's targeted at you but doesn't include you may not be worth sitting still for. Those are things that make me believe that this is a one-way change. Because four year olds, the people who are soaking most deeply in the current environment, who won't have to go through the trauma that I have to go through of trying to unlearn a childhood spent watching Gilligan's Island, they just assume that media includes consuming, producing and sharing.

It's also become my motto, when people ask me what we're doing--and when I say "we" I mean the larger society trying to figure out how to deploy this cognitive surplus, but I also mean we, especially, the people in this room, the people who are working hammer and tongs at figuring out the next good idea. From now on, that's what I'm going to tell them: We're looking for the mouse. We're going to look at every place that a reader or a listener or a viewer or a user has been locked out, has been served up passive or a fixed or a canned experience, and ask ourselves, "If we carve out a little bit of the cognitive surplus and deploy it here, could we make a good thing happen?" And I'm betting the answer is yes.

Thank you very much.

For a good presentation

Books:

• Presentation Zen: Simple Ideas on Presentation Design and Delivery - Garr Reynolds
• The Art and Science of Creating Great Presentations - Nancy Duarte
• The Presentation Secrets of Steve Jobs - Carmine Gallo
• Confessions of a Public Speaker - Scott Berkun

Blogs:

• presentationzen.com

Tools:

• PowerPoint (Windows)
• Keynote (Mac)
• SlideRocket
• Prezi

Videos:

Something to suggest?

Let me know here (comment) or give me a tweet (@albybisy)

Andrea Pontremoli : La forza del sogno

"La forza del sogno è quella di disegnare qualcosa dove agli altri vogliono essere" Andrea Pontremoli

Via: venicesessions.it

Unlearn Your MBA

David Heineimeier Hansson (@dhh), the creator of Ruby on Rails and partner at 37signals in Chicago, says that planning is guessing, and for a start-up, the focus must be on today and not on tomorrow. He argues that constraints--fiscal, temporal, or otherwise--drive innovation and effective problem-solving. The most important thing, Hansson believes, is to make a dent in the universe with your company.

The Peter Principle

The Peter Principle is the principle that:

"In a Hierarchy Every Employee Tends to Rise to His Level of Incompetence."

It was formulated by Dr. Laurence J. Peter and Raymond Hull in their 1969 book The Peter Principle, a humorous treatise which also introduced the "salutary science of Hierarchiology", "inadvertently founded" by Peter. It holds that in a hierarchy, members are promoted so long as they work competently. Sooner or later they are promoted to a position at which they are no longer competent (their "level of incompetence"), and there they remain, being unable to earn further promotions. This principle can be modeled and has theoretical validity. Peter's Corollary states that "in time, every post tends to be occupied by an employee who is incompetent to carry out his duties" and adds that "work is accomplished by those employees who have not yet reached their level of incompetence".

Via: wikipedia

The Economic Motivation of Open Source Software: Stakeholder Perspectives

Open source software has changed the rules of the game, impacting significantly the economic behavior of stakeholders in the software ecosystem. In this new environment, developers strive to be committers, vendors feel pressure to produce open source products, and system integrators anticipate boosting profits.

1. INTRODUCTION

The advent of open source software has produced more than lower software costs for users. It has also created major changes in the economic interaction among players in the software ecosystem.

For many, open source embodies a specific approach to software development---even a lifestyle. But it's also sound business strategy. Ron Goldman and Richard Gabriel suggest that companies should use open source software to grow their user communities and build an ecosystem around their products and services.

Open source software is typically free and comes with the source code needed to adapt it to users' needs. Most open source licenses let users redistribute the software, including possible changes, and charge for redistribution as long as source code changes are publicly available (www.opensource.org).

There are two types of open source software. Community open source is software that a community develops. Rather than a single corporate entity owning the software, a sometimes broad community of volunteers determines which contributions are accepted into the source code base and where the software is headed. Individual developers, the committers, and not a specific company, make decisions about the software, as in the case of the Apache Web server (httpd.apache.org)

Commercial open source is software that a for-profit entity owns and develops. The company maintains the copyright and determines what is accepted into the software code base and what to implement next, as in the case of MySQL and its MySQL database (www.mysql.com).

Prior work on community open source economics focused mostly on labor economics, that is, the frequently surprising amount of volunteer work that goes into open source software. Eric Raymond notes that developers contribute to open source projects for the personal gratification that comes from increasing their reputation among peers. Ernan Haruvy and his colleagues reached similar conclusions in their empirical study.

Joshua Lerner and Jean Tirole, meanwhile, argue that developers contribute to document their technical capabilities and improve job prospects with future employers. And Karim R. Lakhani and Robert G. Wolf report that enjoying their work is a key intrinsic motivation for developers to contribute to open source projects, although survey respondents also revealed that financial incentives are important.

While this explains some of the volunteer work, it doesn't explain why companies today employ people who contribute to open source projects on company time. Il-Horn Hann and colleagues found that the salaries of Apache Software Foundation project contributors correlated positively with the contributor's rank in the Apache organization. They therefore concluded that employers use a developer's rank within the foundation as a measure of productive capabilities.

In the following, we discuss the economic motivation of system integrators, independent software vendors, and software developers to explain these seeming contradictions. A particular focus is on their effects on developer careers.


2. SYSTEM INTEGRATOR PERSPECTIVE

Large system integrators, or solution providers, stand to gain the most from open source software because they increase profits through direct cost savings and the ability to reach more customers through improved pricing flexibility. Every dollar a system integrator saves on license costs paid to a software firm is a dollar gained that the customer might spend on services.

2.1 IT solutions demand curve

Customers typically want information technology providers to deliver "solutions." A solution solves a customer's IT problem, freeing the customer to focus on business rather than IT. A comprehensive solution comprises hardware, software, and services. Indeed, the IT industry earns its living by removing or reducing customers' IT worries.

System integrators deliver solutions by selling a stack of hardware, software, and services as one product. That allows the customer to talk to one company, rather than many. Figure 1a illustrates this stack together with the customer demand curve.


Figure 1. IT solutions demand curve. (a) System integrators sell a stack of hardware, software, and services. (b) Integrators can charge customers similar prices even if they use open source software.

The demand curve shows how many customers are willing to buy the system integrator's solution at a given price. On the Y axis is the customer's cost to purchase a solution, and on the X axis is the number of customers who are willing to pay for that solution at the given price. The form of the demand curve varies depending on what is being sold. However, in general, the demand curve is downward sloping: The lower the price, the more customers are willing to buy.

A system integrator's profits depend on which of the stack's components it owns and which it must buy. Usually, a system integrator's stronghold is services, which puts together the hardware and software pieces to meet the customer's need. However, if the system integrator owns only the services component, it will have to pay other companies for the software and the hardware and thereby share revenue, leaving less profit for itself.

It's therefore in a system integrator's interest to acquire hardware and software as cheaply as possible. Open source software, if an option, is typically much cheaper than closed source software, hence its use increases profits for the system integrator.

Figure 1b illustrates how with stable supply and demand, more money is made in the services part of the value stack if software cost goes down.

Software cost savings aren't easily passed on to customers for two reasons: First, customers tend to care about the whole product rather than individual components; second, large system-integration projects are complex and new competition doesn't spring up easily. Thus, system integrators can maintain their prices.

While this is one good reason for system integrators to support open source software, there's another equally compelling reason for them to support and contribute to open source software.

2.2 Business growth

The simple value stack that Figure 1 illustrates suggests that system integrators charge customers only one price. In reality, the system integrator can charge varying prices for a total solution to a prospective customer's problem. Only one thing is certain: A system integrator will want to at least cover its costs.

The price charged per customer that Figure 1 shows can be split into the system integrator's service cost, plus the markup or margin needed to make a profit. If the system integrator owns just the services part of the stack, the cost for providing that service defines the lower price limit for the work. In a reasonably competitive market, the system integrator will accept deals above this limit if it has the resources.

This limit, together with the demand curve, determines the maximum number of customers the system integrator can sell to and take on, as Figure 2a illustrates.

Switching from more expensive closed source software to less expensive open source software increases the profits of a sale through the money saved on the software. It also reduces the lower price limit for possible deals and puts a new set of more price-sensitive customers within reach. Not only does open source software improve profits on the original individual sales, it also increases the total number of potential customers.

Figure 2b shows how a switch from closed source to open source software results in more potential customers. And more potential—and presumably satisfied—customers mean higher sales and profits. The total profit is represented as the area of the gray triangle under the demand curve, showing the increase in profits when moving down the curve. Since in reality a system integrator might own many of a total solution's components, including software and hardware, more customers mean more profits through these components as well.


Figure 2. Sales margins and number of customers. (a) The lower price limit determines the customers the system integrator takes on. (b) Switching from closed source software to open source software can result in more customers and higher profits.

2.3 Pressures in the IT value stack

If it were up to the system integrators, all software would be free (unless they had a major stake in a particular component). Then, all software license revenue would become services revenue. To this end, I believe that system integrators prefer community open source over commercial open source. Only community open source software prevents vendor lock-in.

Community open source ensures that prices for software support are subject to market forces rather than one owning corporation. Community open source is a strategic weapon for system integrators to squeeze out proprietary as well as commercial open source software vendors.


3. SOFTWARE VENDOR PERSPECTIVE

Independent software vendors provide only a few software products, sometimes just specializing in one. Understanding the independent software firm strategy requires comparing open source software and closed source software cost and pricing.

3.1 Software cost and pricing

Why is open source software typically much cheaper than closed source software? In a closed source business, most of the investment in new software comes from shipping the first copy. The initial investment is recouped with increasing sales. The additional cost of producing and selling another copy is small, consisting of producing another CD or allowing for another download plus providing the (frequently minimal) free support that comes with a software license.

As the number of copies sold increases, the average cost per copy declines and the profits rise. Figure 3a illustrates the long-run average cost curve for a single software product.


Figure 3. Differences in cost and pricing. (a) and (c) show a similar average cost per unit for closed source and open source software, while (b) and (d) indicate the relationship between price and number of units sold for open and closed software.

The more mature a market for a specific software component, the higher the investment in the existing products, the higher the barrier to entering this market, the more established the existing players, and the more stable the price for the software component. A common scenario is "the 800-pound gorilla" firm that dominates a market and is surrounded by smaller players catering to market niches .

In such a market, the leading software vendor sets a price that maximizes its profits. Since the market is fairly transparent, the vendor can set just one pricing schedule, offering the product to different customers at the same price. (This is in contrast to the highly individual deals of large system integrators.) The result is frequently a flat price, as Figure 3b shows. Remarkably, the price of the proprietary closed source software doesn't directly depend on the actual cost incurred to develop, maintain, and provide the software.

The profit-maximizing price is largely independent of cost; the cost provides only a lower limit. Competition that drives prices closer to cost can't spring up easily due to the large initial investment a software product requires.

In a community open source situation, no such market-entry barriers exist. Given the right license, anyone can set up a company and start selling software. What the company will sell, of course, isn't the software itself, but its provision, maintenance, and support.

Because anyone can enter an attractive open source market, competition is fierce, and pricing will be based on markup over cost. If the markup is too high, new companies will enter the market; if it's too low, companies will leave the market. Moreover, the more mature the product, the lower the overall price.

Figure 3c shows the total cost of developing open source software. The total cost and the resulting average cost per copy sold is mostly the same as for the closed source solution. The main difference, of course, is that the different contributing companies now share this cost.

Figure 3d illustrates the pricing of open source software from a single firm's perspective. Because of the competition, the price charged for providing the software plus support is based on markup over cost. (The graph merging various dimensions into one 2D graph simplifies the situation, although the basic argument holds.)

Different firms will have different costs depending on their share of contributions to the open source project. However, with increasing project share, the company can charge higher prices because customers are likely to receive better service. The basic relationship remains unaffected: Price is markup over cost and varies depending on cost.

Customers love this situation because prices are substantially lower than in the closed source situation. System integrators love the situation even more because they can squeeze out proprietary closed source software.

3.2 Generating software profits

A system integrator can increase its profits if it reduces the software cost. By reducing software cost, it can move down the demand curve and sell to more customers.

Closed source software is the main obstacle to doing this: It cuts into profits on an individual sale and reduces overall pricing flexibility. Hence, system integrators have a high interest in turning closed source software markets into software markets with at least one viable community open source product.

Before the advent of open source software, entering an established and well-defended market was a risky proposition. With increasingly well-understood open source processes, setting up an open source project competing with an established closed source market leader's product is much less risky and carries a significantly higher chance of success than before. But it's not just a specific system integrator that will want to do that. It's pretty much everyone who isn't the closed source market leader.

To understand this, put yourself in the shoes of the CEO of an also-running traditional closed source company. At one point, it's becoming clear that you won't be the leading firm in your market and that your company's profits will come from market niches at best. Neither you nor your investors are happy with the projected return on investment.

Your best option now is to open source your product. You might be reducing the market's overall return on investment, but at least you'll have a second chance at satisfying your own investors by making your company a successful open source business. You'll be in good company. With a proper license for your open source product, you might well receive help from the system integrators, customers, and software vendors higher in the IT stack.

Now, assume you're the CEO of the market-leading company in some space. Thinking ahead, you have to assume that either a competitor will open source its product or that a system integrator will instigate an open source product—or both. The proactive answer to this scenario is to open source your product, even though you're the market leader and would win big in the old closed source world. But it's better to win in an open source world than not to win at all.

These two thought experiments show that community open source software has a high chance of taking new markets early on. Only strong intellectual property protection or other competitive advantages might lead a closed source company to win and keep a new market. Leaders in established markets might be able to defend their positions for a long time. They tend to dig in with complex products, established processes, customer data lock-in, and many other positional advantages. Still, open source might well prove to be disruptive enough to conquer even these markets.

3.3 Commercial Open Source

With such gloomy prospects for closed source businesses, independent software vendors have sought business models for harnessing open source software's benefits while gathering some of the profits of a closed source business. The answer is commercial open source.

The key differentiator between community open source and commercial open source is whether a community or a single entity like a corporation holds the power to make decisions about the project.

Commercial open source software is typically available for free to nonprofit users. Sometimes commercial use is free as well. Usually, companies make money by providing support services. Sometimes they make additional money by selling proprietary software enhancements.

Like community open source, commercial open source is available in source code form. Unlike community open source, however, one company controls commercial open source. This way, commercial open source software can gather some of the benefits of community open source: faster adoption, free and speedy user feedback, and possibly volunteers' code contributions. This approach is mostly a marketing strategy, however, because the company that owns the software still must do the development. Hence, the company must employ and pay the software's developers.

During the early days of an open source project, this is an advantage, as the company can provide clear direction and muster more resources than community open source projects typically can. As the project matures, this can turn into a disadvantage, as a competing community open source project might have more resources at hand in the form of volunteers.

The upside for the owning company is that little open source competition can spring up for its product. However, system integrators have a strong interest in providing alternatives to proprietary software, and this applies to commercial open source as well. Hence, this business model is likely to experience the same pressures as proprietary software.

3.4 Open source service firms

If it isn't possible to be a profitable closed source business, what does it mean to be a successful open source business? The market's answer is the open source service company, which comes in at least two kinds. One provides first-level support and implementation services; the other provides second-level support, training, and development services.

Clients of the first kind of firm are typically IT users who employ the firm's services to put the open source product into place in their IT operations. Clients of the second kind of firm typically need to get trained on the product or need to have a technical problem fixed that they can't handle themselves.

The strength of a service business usually lies in its ability to:

• recruit and retain the right people,
• reliably set up and execute specific service processes, and
• bring to bear expert domain knowledge and unique intellectual property.

In the open source situation, this is usually labor economics. Technical skills around the open source product are a key part of determining an employee's value to a firm. Anyone who's smart enough can develop these skills because the open source software is available to people outside the firm.

Hiring and firing becomes easier because there's a larger labor pool to draw from, and switching costs between employees are lower compared with the closed source situation. Given the natural imbalance between employers and employees, this aspect of open source is likely to increase competition for jobs and drive down salaries. Lower salaries aren't as much of an advantage to the software vendor as might be expected because in the more transparent and competitive open source situation, such cost savings are likely to be (at least partially) passed on to customers.

3.5 The need for committers

An employee's position in the open source project is another key part of his or her value to a firm. The organizational setup varies between open source projects, but in some form, people always play user, contributor, and committer roles. Users use the software, contributors contribute in some form, and committers decide what contributions to accept into the project.

Figure 4 illustrates how a developer might progress through the ranks of a community open source project: A committer typically promotes a user to a contributor role implicitly by accepting the user's contribution into the software. A contributor is typically promoted to a committer position explicitly, through a prior vote of the existing group of committers and a subsequent public announcement of the contributor's ascension to committer status.


Figure 4: Positions and promotions. In open source projects, users are implicitly promoted to contributor status, while contributors are explicitly promoted to committer status.

Committers determine where the open source project is headed, strategically and on a day-to-day basis. They can typically resolve technical problems faster than noncommitters, and have high visibility to the user community. Most projects are set up so there's only a small inner circle of committers, a larger set of contributors, and an even larger user community.

For an employer, the value of employing a committer is manifold. Through the committer, an employer

• gets problems with the open source software fixed faster and better,
• can better align company strategy with the open source project and vice versa,
• appears as a more attractive employer than competitors who don't employ a committer, and
• has higher visibility with the user community and can reach out more effectively.

A major goal of any open source service company is to convert freeloading users into paying customers. A committer's visibility with the user community is an important marketing advantage that an employer can use to support this goal.

Thus, committers have a strong negotiation position with their employers. Employing a committer is important for a first-level support and implementation services company, and it's critical for a second-level support service company.


4. THE EMPLOYEE PERSPECTIVE

Open source software and service businesses make life more complicated for employees. Employees build up less firm-specific knowledge simply because there's less of it. People from the outside can replace them more easily. At the same time, an employee's day-to-day work improves non-firm-specific knowledge of an open source project that can be taken to another employer. So a developer who is fired can find a job faster than before.

4.1 Benefits of being a committer

An employee who is a committer is likely to earn higher compensation. Hann and colleagues have empirically verified this for committers to Apache Software Foundation projects.

At any time, the committer-employee can credibly threaten to leave the company, taking significant power and reputation away from the current employer. Employers often pay premium salaries just to employ prominent committers.

But how do you become a committer? Community open source projects tend to be meritocracies, judging developers by their social and technical contributions. In contrast, a company owning some commercial open source gives committer status to its employees (and takes it away) as it sees fit.

Consequently, it makes little sense for the economically rational software developer to invest time in commercial open source. The value these developers create is tied to the product and the owning company. Unless the product is in wide use or the developer wants to work only for this one company, it makes more sense to invest time in a community open source project.

4.2 How to become a committer

Developers who start projects immediately become committers. However, they now face the task of creating a successful project out of nothing. This is a highly entrepreneurial activity: Developers must promote their project while doing the actual programming work, understanding that the outcome is uncertain.

It's more common to join an existing open source project. Assuming a fair and transparent promotion process, the two main criteria that will get a developer promoted from contributor to committer are:

• the developer's social and technical abilities, and
• demonstrated commitment to the open source project.

This is what most project Web sites state and what developer surveys have revealed. However, these surveys rely on what developers say they do, and actual behavior could vary from what people believe motivates them.

Because being a committer can have clear financial benefits, keeping the group of committers small is in the economic interest of a committer to a successful project. Not doing so would dilute the committer's value to current and future employers. At least this would be an economically rational person's train of thought.

Counteracting the existing committers' economic interests is the need and desire to build a working community. Also, the participants in a new project are likely to appreciate every helping hand while a mature project might not need any additional committers.

Thus, the following forces possibly influence a developer's promotion:

• the economic self-interest of the group of existing committers,
• the committers' philosophical convictions on running the project, and
• the project's need for more committers.

In many ways, investing in an open source project is like joining a startup. The earlier a developer joins, the higher the risk of the project not working out but also the more likely the ascension to committer status. The later a developer joins, the lower the risk, but also the lower the chances of becoming a committer any time soon.

The window of opportunity is small for those aspiring to achieve committer status in an important open source project. With the ongoing commercialization of open source, many current projects expect a committer to work full-time on the open source project. Otherwise, committer status wouldn't be granted. This, for example, is what the Eclipse project Web site states about its core projects (www.eclipse.org). However, a company is likely to let an employee work full-time on an open source project only if that person is already a committer; otherwise how many of the benefits of its contributions the company would reap is uncertain.

A developer who chooses the right project can gain and maintain a position that will increase salary-negotiation power and job prospects. The developer will enjoy those benefits as long as the project is of significance to potential employers.

Open source reinforces the trend toward employees becoming "free agents." Committers who rationally follow their economic interests are likely to be more loyal to the open source project than to their current employer because that's where their market value lies. That results in a more fluid job market where developers can be expected to move around more freely and more frequently than in the past.


5. CONCLUSIONS

Open source software has enabled large system integrators to increase their profits through cost savings and reach more customers due to flexible pricing. This has upset existing ecosystems and shuffled structural relationships, resulting in the emergence of firms providing consulting services to open source projects. This new breed of service firm in turn lives or dies by its ability to recruit and retain appropriate talent.

For such talent, in particular for software developers, life has become more difficult and exciting at once. Developers face new career prospects and paths, since their formal position in an open source project, in addition to their experience and capabilities, determines their value to an employer. Economically rational developers strive to become committers to high-profile open source projects to further their careers, which in turn generates more recognition, independence, and job security.

Via: dirkriehle.com

The Keynesian beauty contest

A Keynesian beauty contest is a concept developed by John Maynard Keynes and introduced in Chapter 12 of his work, General Theory of Employment Interest and Money (1936), to explain price fluctuations in equity markets.

Overview
Keynes described the action of rational agents in a market using an analogy based on a fictional newspaper contest, in which entrants are asked to choose a set of six faces from photographs of women that are the "most beautiful". Those who picked the most popular face are then eligible for a prize.
A naïve strategy would be to choose the six faces that, in the opinion of the entrant, are the most beautiful. A more sophisticated contest entrant, wishing to maximize the chances of winning a prize, would think about what the majority perception of beauty is, and then make a selection based on some inference from their knowledge of public perceptions. This can be carried one step further to take into account the fact that other entrants would each have their own opinion of what public perceptions are. Thus the strategy can be extended to the next order, and the next, and so on, at each level attempting to predict the eventual outcome of the process based on the reasoning of other rational agents.

It is not a case of choosing those [faces] that, to the best of one’s judgment, are really the prettiest, nor even those that average opinion genuinely thinks the prettiest. We have reached the third degree where we devote our intelligences to anticipating what average opinion expects the average opinion to be. And there are some, I believe, who practice the fourth, fifth and higher degrees. (Keynes, General Theory of Employment Interest and Money, 1936).

Keynes believed that similar behavior was at work within the stock market. This would have people pricing shares not based on what they think their fundamental value is, but rather on what they think everyone else thinks their value is, or what everybody else would predict the average assessment of value is.

Subsequent theory
Other, more explicit scenarios help to convey the notion of the beauty contest as a convergence to Nash Equilibrium. For instance in the p-beauty contest game (Moulin 1986), all participants are asked to simultaneously pick a number between 0 and 100. The winner of the contest is the person(s) whose number is closest to p times the average of all numbers submitted, where p is some fraction, typically 2/3 or 1/2. If p<1 p="1">
In play of the p-beauty contest game (where p differs from 1), players exhibit distinct, boundedly rational levels of reasoning as first documented in an experimental test by Nagel (1995). The lowest, `Level 0' players, choose numbers randomly from the interval [0,100]. The next higher, `Level 1' players believe that all other players are Level 0. These Level 1 players therefore reason that the average of all numbers submitted should be around 50. If p=2/3, for instance, these Level 1 players choose, as their number, 2/3 of 50, or 33. Similarly, the next higher `Level 2' players in the 2/3-the average game believe that all other players are Level 1 players. These Level 2 players therefore reason that the average of all numbers submitted should be around 33, and so they choose, as their number, 2/3 of 33 or 22. Similarly, the next higher `Level 3' players play a best response to the play of Level 2 players and so on. The Nash equilibrium of this game, where all players choose the number 0, is thus associated with an infinite level of reasoning. Empirically, in a single play of the game, the typical finding is that most participants can be classified from their choice of numbers as members of the lowest Level types 0, 1, 2 or 3, in line with Keynes' observation.

In another variation of reasoning towards the beauty contest, the players may begin to judge contestants based on the most distinguishable unique property found scarcely clustered in the group. As an analogy, imagine the beauty contest where the player is instructed to choose the most beautiful six faces out of a set of hundred faces. Under special circumstances, the player may ignore all judgment-based instructions in a search for the six most unique faces (interchanging concepts of high demand and low supply). Ironic to the situation, if the player finds it much easier to find a consensus solution for judging the six ugliest contestants, he may apply this property instead of beauty to in choosing six faces. In this line of reasoning, the player is looking for other players overlooking the instructions (which can often be based on random selection) to a transformed set of instructions only elite players would solicit, giving them an advantage. As an example, imagine a contest where contestants are asked to pick the two best numbers in the list: {1, 2, 3, 4, 5, 6, 7, 8, 2345, 6345, 9, 10, 11, 12, 13}. All judgment based instructions can likely be ignored since by consensus two of the numbers do not belong in the set.

Via: wikipedia