Committocracy as an alternative for conflict resolution in OSS projects

I my previous post I presented an unsparing criticism of Apache's voting procedures. In this post, I describe a new voting procedure where each committer's voting rights derive from the number of commits made by the committer in question. To avoid rewarding micro-commits, only a single commit-point is awarded per day for one or more commits. No points are awarded for a committer who makes no commits on a given day.

Non controversial questions are settled by consensus. However, whenever a decision cannot be reached by unanimous agreement, a vote is called for. The commit-point for or against a motion are summed, the total accumulated commit-points determining the outcome.

Such a system endows a collaborative software project, open source or not, to reach agreement in a timely and orderly fashion. It is much less vulnerable to disruption by unscrupulous participants than Apache's current voting procedures. I designate an organization where voting power derives from the number of commits made by an individual, typically a software developer, as a committocracy.

Feasibility

Given any reasonably-open version control system, it should be fairly easy to write software which assigns commit points to each committer. During a vote it is a matter of simple arithmetic to tally the commit-points expressed for or against a motion.

In the case of git, the following command can be used to compute the commit-points accumulated by Alice.

git log --format='%ad %an' --date=short|uniq|grep Alice|wc -l

Worse of both worlds or best of both worlds?

A committocracy is less efficient than the BDFL model for decision making, and compared to the Apache-way, it grants less power to newcomers. However, a committocracy is a fair system in the sense that the same rules apply to all. Today's committer with the most committer-points can be different than that of tomorrow. Moreover, compared to the Apache-way, a committocracy drastically reduces the risk of a project going haywire after admitting a new member. As a corollary, a project can safely reduce the wait-and-see period preceding the admission of new committers. Thus, newcomers may be granted committership status more quickly.

Psychological effects

Immediately after electing a new committer, accepting him or her as an equal member of the community has powerful positive effects on the psyche of the electee. In a minority of cases it results in the electee becoming a major contributor of the project. However, this positive effect is often short-lived. After a few months of modest activity, it can even transform itself to sense of entitlement which can be devastating at the hands of a less-than-scrupulous committer wielding veto powers. Fortunately, most people are fairly decent and do not abuse their rights in any blatant way.

In a committocracy, voting power accrues with each day involving a commit. As the entitlement of the committer grows with each contribution, the positive psychological effects of belonging to the community may be longer lasting, in particular because accrual in voting rights it is deserved, i.e. based on "merit". However, since I am not aware of any committocracy in existence, I can only speculate on the longer duration of said positive effects.

Is a committocracy a meritocracy?

No, not exactly. Granting one voting point per day rewards participants most committed to the project over a lengthy period. It does not directly take into account the value of each commit. In a very general sense, granting veto rights to all committers can also be considered as a meritocracy because it rewards participants with most endurance in arguments. It is just a different, and in my opinion less pertinent, measure of merit.

Does it matter?

What is the point of a committocracy , since it obviously does not perfectly capture the notion of merit? Moreover, why change the current voting system if only a few people complain about it?

While a committocracy is not a perfect or even a very good measure of merit, it is still a much better measure than a 0 or 1 switch correlative with veto powers. Since OSS contributors surrender so much of their independence to the OSS organization, in this case the ASF, the organization must ensure that its rules are as just as possible.

The forces and vulnerabilities of the Apache model

The initial title for this article was "Why the Apache model sucks". It would have been a catchier title but would taint my arguments with triviality. But it was the first title that came to my mind and you should be aware of that.

I have written about Apache in the past past and the present post is a rehash with a slightly different emphasis. Before laying further criticism at the altar of the "Apache Way", it must be mentioned that Apache is one of the most open and transparent organizations I know of. Transparency comes at a price. If one is allowed or even encouraged to voice criticism, the voices of the critics may drown the successes of the organization so that the state of affairs may seem bleaker than it is really is.

On the other hand, an organization can be wildly successful even if some of its governing rules prove to be counter-productive on the long run. As a very extreme example, in the Ottoman-empire, one of the most successful and relatively recent empires in history, whenever a new sultan acceded to power he would proceed to kill all his brothers. Not only fratricide was authorized, in the absence of clear succession rules other than the survival of the fittest, the people actually expected the new ruler to kill all his brothers so that the strongest ruler could emerge. The rulers who chose to spare the lives of their brethren were accused of not getting the Ottoman-way. Suleiman the Magnificent went as far as killing his own son Mustafa after bogusly charging him of treason.

One of the the core tenets of Apache is meritocracy. If Apache is a meritocracy, it is a lousy one and this has consequences. If some developer, say David, shows interest in some Apache project, say Prokayotae, and contributes by submitting patches, or by responding to users on the mailing list for a period of time, usually three months or more, then he will be awarded committership. As Prokayotae committer, David can veto decisions of the project – yes, that is veto with a 'v'. After a few years of good behavior, David could even be co-opted to become a member of the Apache Software Foundation. Being a member is a nice honorary title but does not carry further entitlements at the project level. As a member David can indeed vote for the Apache Board during elections but that is mostly inconsequential at the project level.

Coming back to Prokayotae, the newly elected committer David wields the same voting power as Carol, who has been actively contributing to Prokayotae over several years. In most cases David will be a reasonable individual who play nice and will naturally defer to Carol's and other committers' opinions by virtue of common decency and their past involvement in Prokayotae. However, while most new developers are reasonable individuals and make every effort to play nice, some people are less reasonable. Even a modestly successful project will have over a dozen committers elected over its lifetime. Thus, the probability of electing an "unreasonable" individual increases over time, and I'd venture to say, approaches certainty.

Let Ursula be such an unreasonable person elected as a Prokayotae committer. Ursula maybe otherwise a nice person overall and a valuable member of the community she lives in. But, as a fervent believer in Thor, one night Ursula has a dream where Thor ordains her to oppose the pernicious influence of Carol over the Prokayotae project. Of course, the motivations for Ursula's opposition to Carol may have origins other than Thor appearing to her in a dream. The potential reasons are innumerable. It may be as simple as a design principle lambda that Ursula reveres but which Carol reveres less.

Invested with this new mission in life, Ursula begins to challenge Carol by vetoing her commits or merely intimating that she will veto them based on the lofty lambda design principle. Were Ursula's objections patently silly, she will be revealed as a fool. For example, invoking Thor's appearing in her dreams as a justification for her opposition will expose her to ridicule. However, any reason which is based on some apparently reasonable pretext, often adherence to a lofty principle, provides sufficient cover for long-term disruption. Every developer knows that the IT industry can claim more than its fair share of high and often contradictory principles. Ursula will make convoluted arguments, misrepresent facts and argue endlessly. Given that each new commit takes ages in endless arguments, further development in Prokayotae will be severely disrupted and may even cease.

Carol who has invested heavily in Prokayotae and got along well with the other developers may be shocked and ill-prepared to deal with Ursula's disruptive interventions. Carol may be further shocked to see other committers sitting on the fence with respect to Ursala's arguments cloaked under an important design principle such as lambda. At this stage, Carol must tread with utmost care because she formally has the same voting rights as any other committer, including Ursula. If Carol dares to claim that her arguments should carry more weight in light of the volume of her past contributions, she will immediately lose her credibility.

The Apache culture aggressively enforces its egalitarian model. Carol, even if she effectively led the project for several years, is not allowed the title of project leader or project manager. Author tags in source code are also frowned upon with the Apache board having formulated a statement discouraging their use. Hints at unequal merit are met with condescension and/or alarm.

As a result, if Carol tries to explain the injustices of the Apache way she will be branded as clueless and subsequently ignored. Being ignored is Apache's equivalent to being ostracized in the pre-Internet age.

As most people who face injustice and have a choice, Carol will leave Prokayotae as I have left log4j to start the SLF4J and logback projects. Leaving log4j was one of the most traumatic experiences of my life. Trustin Lee, of Netty and Mina fame, had an analogous experience. He apparently could not bear to see his work being vetoed over a triviality.

Those favoring the current voting system, while recognizing its lopsidedness, argue that letting everyone having an equal voice fosters communication between developers. Admittedly, the current set up is conducive to collaboration as progress can only be achieved after reaching consensus. As a corollary to this line of reasoning, letting all arguments be heard and all objections raised at every step of the development process must surely lead to best possible software.

Unfortunately, in the absence of a fair conflict resolution mechanism, having lots of ideas floating around emanating from different people does not lead to the emergence of the best software design insofar as it promotes bickering and political paralysis. In such a system where development can be easily disrupted, it is not the best ideas that win but the ideas advocated by those possessing the most stamina.

Instead of trying to learn from past failures which open discussion is supposed to encourage, Apache forges on in the path of egalitarianism. As time passes, I see attempts at institutionalizing egalitarianism instead of recognizing its inherent injustice. For the sake of a bogus ideal, the Apache-way expects selflessness on the part of the developer in the same way that the catholic church expects celibacy from its priests. If egalitarianism is really at the core of the Apache way as an absolute value, then the Apache way sucks. Yay!

While the one person one vote principle applies to a democracy in order to run a country for the benefit of all, the one committer one veto principle is ill-suited in a purported meritocracy such as Apache. If it must be "one committer one veto", then the word meritocracy cannot be honestly ascribed to Apache. It it would be most appropriate for the ASF to stop misrepresenting itself as a meritocracy, at least not without clearly defining the meaning of merit.

The Apache culture fails to recognize that project participants may have drastically different investments in a project. Allowing a person with near-zero involvement in a project to hold the same weight as a person with 10'000 hours of investment is to date an essential part of the Apache-way. According to the prevailing Apache culture, if you don't completely agree with this premise, body and soul, then you just don't get it.

I hope that this blog entry will incite OSS organizations, including the ASF, to adopt fairer decision making procedures. If that does not happen, before joining such an organization, developers should at least be aware of the inconveniences associated with lack of fair and orderly decision making mechanisms at the project level. If knowing these inconveniences, a developer decides to join anyway, then it will be the result of an informed decision.

Encouters in Groovy I

You can take any .java file and rename it .groovy and the result will be valid Groovy. Compared to other contenders in the new-Java space such as Scala or Clojure, the syntactical-backward compatibility offered by Groovy is undoubtedly an important and possibly a decisive advantage.

Let me demonstrate. Here is a very simple unit test measuring the performance of a trivial arithmetic operation.

// file MyTest.java
package ch.qos;
import org.junit.Test;
public class MyTest {
  static int LEN = 100*1000;

  @Test
  public void smoke() {
    // let the JVM warm up
    loop();
    loop();
    double result = loop();
    System.out.println("Average duration per operation: "+result+ " nanoseconds");
  }

  double loop() {
    long start = System.nanoTime();
    double sum = 0;
    for (int i = 0; i < LEN; i++) {
      sum += i*1.0;
    }
    long end = System.nanoTime();
    return (end - start) / LEN;
  }
}

Running the above test yields:

Average duration per operation: 3.0 nanoseconds

As mentioned earlier, a valid Java class is also a valid Groovy class. So renaming "MyTest.java" as "MyTest.groovy" results in a valid Groovy class. With JetBrains IDEA which provides pretty nice Groovy support, I can run "MyTest.groovy" as any other junit test. Here is the result:

Average duration per operation: 843.09 nanoseconds

Lo and behold, the same code runs 280 times slower when compiled as a Groovy class than its Java counterpart. If I were blogging for a sensation-driven news organization with an anti-Groovy agenda, I would now prematurely claim the death of Groovy and stop writing.
As I don't work for a sensation-driven news organization nor have an anti-Groovy agenda, I will try to mitigate the preceding results.
The code generated by Groovy works on objects instead of primitive types. For example, the 'i < LEN' check is done by invoking the compareLessThan() method in the
the ScriptBytecodeAdapter class part of the groovy runtime. This method operates on objects instead of the primitives types. I suspect that the dynamic-nature of Groovy forces it to invoke methods flexible enough to deal with untyped objects, instead of invoking more trivial byte code which the hot-spot compiler is pretty masterful at optimizing -- but that's just my hunch.

We can actually improve the performance of the loop by using more idiomatic groovy. Modifying the iteration from

for (int i = 0; i < LEN; i++) {
  sum += i*1.0;
}

to

for (i in 1..LEN;) {
  sum += i*1.0;
}

brings down the average duration per operation from 843 to 675 nanoseconds. By avoiding the integer to double conversions we can drastically improve performance. Here is the modified iteration:

for (double i in 1..LEN;) {
  sum += i;
}

Surprisingly enough, this last optimization brings the average duration to 50 nanoseconds, a 17 fold improvement from the initial non-idiomatic version of the code running at 843 nanoseconds per operation.

We are still far from the 3 nanoseconds obtained from initial .java version of the code. Perhaps the code can be further optimized and close the gap with the original .java version.

Groovy is indeed slower than Java in tight loops. However, it so happens that the performance of most applications is I/O bound, so the practical performance impact of Groovy may be largely offset by the (developer) productivity gains it offers.

In conclusion, while blindly converting .java files to .groovy may result in a catastrophic degradation in performance, a more selective migration can result in significantly better code without serious degradation in performance.