2010 Contest Design

1. Introduction

This detailed design for the Climate Collaboratorium 2010 Contest is organized according to the four key elements of organizational design described by Malone, Laubacher, and Dellarocas (2010): (1) What is being done?, (2) Who is doing it?, (3) Why are they doing it? and (4) How are they doing it?

For this contest, the answers to these questions are as follows:

2. Proposals (What)

For the 2010 contest, proposals will focus on the question: What international climate agreements should the world community make?

All proposals will be grouped as entries under this proposal question.

2.1. What will proposals contain?

Proposals will include at least the following elements:

2.1.1. Emission reduction commitments for major countries or groups of countries

These will be expressed on the Actions tab in the proposal. At a minimum, using the current version of the software, users can enter emission reduction commitments for three major regions of the world (developed countries, rapidly developing countries, other developing countries) and global-level commitments for reducing deforestation and increasing treegrowth. We expect that by the time the contest begins, it will also be possible for users to create and upload spreadsheets that include more detailed breakdowns by smaller regions or individual countries.

(add more details here about how this will work?)

2.1.2. Proposals for how global commitments will be financed

These will also be expressed on the Actions tab in the proposal.

(Need to add details here about how this will work. For instance, what will the input screen look like? Will it be possible to include more detailed breakdowns in spreadsheets?)

2.1.3. Discussions of the other actions needed to make the above commitments feasible

The Description tab will include discussions of the technical, economic, social, political, and other actions needed to make the above commitments feasible.

At present the Description tab starts out completely blank, and users can add any text, links, graphics or other HTML elements. We propose that this tab start out with a template to be defined (see Sample template for Proposal Descriptions). Even though the system would not prevent users from changing anything in the Description field, the template would encourage them to include certain kinds of content, and the contest judging rules should probably require these elements to be included.

2.1.4. Predictions of the physical and economic impacts of these actions (using computer simulation models)

These are automatically generated by the computer simulation models based on the inputs in the Actions tab

2.1.5. Artistic or other representations of the world that would be created by this proposal

See Sample template for Proposal Descriptions

2.2. Models and Sub-proposals

(more description to be added here of how to do model extensions (e.g., with spreadsheets) and sub-proposals)

3. Contest (How)

Even though it is useful to consider many possible proposals for what people can do about climate change, ultimately, we will only be able to carry out one set of global actions. Therefore, since only one proposal will ultimately be used, it is appropriate to have a way of selecting one proposal (or a small number of proposals) from all the proposals that are proposed. According to the framework in Malone, Laubacher, and Dellarocas (2010), that means a Contest design pattern is needed.

In order to have a contest, there must be decision process for selecting winners, and we propose a multi-round process in which expert judges do the first two rounds of screening and then a popular vote select the winners in the last round.

3.1. Rounds

3.1.1. Preliminary round

In this round, any individuals or teams can create proposals in the format described above.

(should there be some way of limiting the number of proposals created at this stage?)

Then a group of expert judges will select some of the proposals to advance to the next stage as semi-finalists, based on the feasibility, novelty, and presentation quality of the proposals (see details of judging criteria below). We believe it is better to let the judges decide, based on the actual proposals submitted, how many semi-finalists to choose.

In order to help the judges find the most promising proposals out of what may be a large number of submissions and to select among proposals that are equally feasible, the system will include a way for people to indicate their support for proposals at this stage.

(need more details here about how this will work. e.g., Users can be "supporters" (or "fans"?) of up to 5 (or some other number of) proposals, and the system will show the number of supporters each proposal has.)

Proposed dates:

• September 1-25 - Proposal creation by teams and endorsement by supporters (should this stage last longer?)
• September 26-30 - Judging

3.1.2. Semi-final round

In this round, teams whose proposals are selected as semi-finalists will continue to develop and improve their proposals.

If new system functionality (such as geographical decomposition of proposals) is not available until this round begins, then this new functionality will be incorporated into proposals at this stage.

(include more detailed proposals for this here? Or specify that different aspects of the proposals will be developed in this stage?)

Community members will still be able to show their support for selected proposals using the mechanisms described above. (should the max number of proposals for which you can show support be decreased in this round?)

At the end of this round, judges will use the same criteria as before to select a smaller number of finalist proposals. As before, judges will decide during the judging exactly how many finalist proposals to select.

Proposed dates:

• October 1-25 - Proposal creation by teams and endorsement by supporters
• October 26-31 - Judging

3.1.3. Final round

The finalist proposals will all be ones the judges believe are feasible and well-presented and that represent a diversity of possible approaches for dealing with climate change.

In this final round, all registered users of the Collaboratorium will be able to vote on the proposals they find most desirable.

(include here any details about how to do this voting, e.g., should vote counts be hidden until the end of the round? should people do preferential rankings? etc.)

Proposed dates:

• November 1-21 - Voting
• November 22 - Announcement of results (Note that this is the Monday before Thanksgiving. should we change this date? For instance, should we announce on the first day of the Cancun talks? or a week earlier Nov 15?)

With this schedule, the results of the contest will be available for use by negotiators at the UN talks in Cancun November 29-December 10.

3.2. Judging criteria

Judges will be asked to evaluate proposals on the following criteria:

• Feasibility of the actions proposed in the proposal. For instance, judges with different kinds of expertise will evaluate the technical, economic, social, and political feasibility of the proposals.
• Novelty of the proposal's ideas. Rather than selecting a number of feasible proposals that are very similar, judges will try to select proposals that represent a variety of different approaches to global warming.
• Presentation quality. Even though presentation quality is the least important of the three criteria, proposals that are well-presented will be favored over those that aren't. Presentation quality includes how well-written a proposal is, how well it uses graphics or other visual elements, and how compelling are its artistic representations of possible future worlds (if any).

There are no explicit weightings for these three criteria, but judges will be asked to use all three criteria to select proposals that are most likely to lead to useful outcomes of the contest overall. As a "tie-breaker", judges may also use the popularity of a proposal as indicated by the number of people who support it.

For example, in cases where a number of proposals are similar, judges will try to pick one or two proposals to represent the whole group. In selecting these representative proposals, judges will take into account the quality of the proposal presentations and the number of people who support the proposals. And in order to increase the diversity of ideas considered in the next round, judges may accept slightly lower levels of feasibility for proposals that include highly novel and interesting ideas.

Judges will also be explicitly asked not to select proposals based on their own personal preferences about which proposals are most desirable. In other words, judges are asked to use their expertise to judge the feasibility of proposals but not their desirability. Judgments of desirability are primarily made in the final stage of the contest by popular vote. For example, a judge should not reject a proposal is technically, economically, and politically feasible, just because the judge feels that the proposal would lead to socially undesirable consequences.

3.3. Selecting judges

Judges with expertise in at least the following areas will be needed:

• physical and biological systems related to climate change and it's likely consequences (atmosphere, ocean, etc.)
• technical and economic systems that generate (or sequester) greenhouse gases
• political and social systems that make choices about legal and regulatory mechanisms, technology use, consumer demand, etc.)
• artistic evaluation of the artistic representations included in proposals

Some of these expert judges may be recruited from our current Expert Advisory Board and Expert Council. But we need to expand these groups or reach outside them for things like:

• political scientists, political analysts, or practicing politicians who can judge the political feasibility of proposals
• artists who can help judge the artistic merit of proposals

(more details needed here about the kinds of judges to recruit and how to recruit them. for example, what incentives will motivate judges to participate?)

3.4. Judging process

In each round, judges will need to make selections among a (potentially large) number of proposals in only a few days, and the proposals themselves may, in some cases, include a lot of material. Therefore, we need to think carefully about how to make this process feasible.

One possibility is to let each judge select one or more assistants to work with them on their tasks. For instance, judges who are professors might select graduate students they already know. Or perhaps we could let (selected?) Collaboratorium Moderators help judges analyze the filter the proposals. Obviously, it will be important to select people for this role who are reliable and who won't try to unduly influence the process based on their personal preferences or inappropriate influence from interested parties.

(more details needed here)

4. Teams (Who)

Individuals may create proposals on their own, but users will be encouraged to form teams to develop proposals. Members of teams whose proposals are eliminated in the first round will be encouraged to join other teams in the semi-final round. If they wish, teams can restrict proposal editing to team members only; alternatively, teams also have the option of letting anyone who is interested edit their proposal.

(need more details here about how the software will support teams. e.g., ability to revert changes made by strangers, ways of showing team members on the proposal(s) they have created, collaboration tools for team members, etc.)

5. Incentives (Why)

Why will people participate in this contest? Using the language of Malone, Laubacher, and Dellarocas (2010), we believe the primary incentives are likely to be Love and Glory, not Money.

5.1. Glory

One very important incentive for people to participate will be recognition ("fame" or "glory"). At one level, people will get some recognition from others seeing their proposals in the system at all. But the recognition will be much greater for people whose proposals are supported by many others and especially for proposals that are chosen as semi-finalists and finalists.

(include more details here about how to increase the recognition people will get. e.g., system features such as badges and leader boards)

5.2. Love

In addition to the possibilities for recognition, people will presumably also be motivated to participate in this contest for a variety of other reasons which we would classify as different forms of "love":

• Interest in the subject of climate change
• Desire to make the world better by helping to solve the problem of climate change
• Desire to express one's own opinions about controversial topics
• Enjoyment of working with fellow team members
• Enjoyment of interacting with a broader community of people interested in climate change
• Excitement of competition in the contest

(include more details here about how to maximize these motivations? for instance, one possibility would be to have organizations like the UN be official partners in the contest and say they will be interested in the results.)

5.3. Money

The third major incentive listed by Malone, Laubacher, and Dellarocas (2010) is money. It is certainly possible to give monetary prizes to the winners of this contest, by for example, finding individuals or organizations who would like to sponsor these prizes. And if these prizes are big enough, they could provide substantial motivation for people to participate.

However, while we have not ruled out the possibility of monetary prizes, our current feeling is that this is probably not a good idea for several reasons:

• If the monetary amounts are small, then they wouldn't provide much motivation and would add significant amounts of complexity to administering the contest. For instance, winning teams would have to figure out how to divide the money among themselves, and the contest administrators (us) would have to raise money, account for its disbursement, verify the identities of people who were receiving money, etc.
• If the monetary amounts are large, they would provide significant motivation, but they would add even more complexity to administering the contest. In addition to the issues listed above, there would be much greater incentives for people to try to cheat and to complain about potential unfairness in the judging and many other aspects of contest administration. While the possibility of significant recognition may cause some of these problems, we believe the risk is much greater if large monetary prizes are involved.
• We believe that it probably "sounds better" to say that this university-sponsored contest for social good does not involve large monetary prizes.

Another possibility we have considered is having a sponsor give a prize which would be donated to non-profit climate-change-related activities (such as charities or research organizations) selected by the winning teams.