The Scala language offers significant improvements over the Java language with traits, higher order functions and type inference among other powerful features. At the same time, Scala still allows for seamless import and use of existing classes written in Java. You can migrate to Scala piecemeal, for example in your test classes at first and then migrate larger and larger chunks of code.
However, there is one aspect to the Scala language which I find deeply annoying. Scala keeps breaking binary compatibility with every new release. In spite of previous promises, compatibility was broken in release 2.7, broken again in release 2.8 and broken yet again in 2.9. As I understand it, Scala language designers are forced to breaking compatibility whenever Scala library traits change in an incompatible way.
When a binary breakage occurs at the language level, the whole ecosystem for the language has to align itself with the new release. This is an extremely painful process affecting all users of the language. Even if a user does not want to upgrade to the latest and greatest Scala release, as long as a single tool, say T, in the tool-chain of the user upgrades and the user upgrades to the new version of T, then kaboom! All other project dependencies need to be upgraded as well.
If you decide to upgrade to the newest version of Scala in your project, you will also need to update every single dependency in your project (written in Scala). If you are lucky and every single dependency has made a release for the latest version of Scala, your project will build fine after the update. Otherwise, if a single dependency has not made the required release, you are left with two relatively unpleasant choices. You can either revert to the previous version of Scala or remove the non-compliant dependency.
If the Scala update was triggered by an IDE update, reverting to the older version of Scala may be particularly painful. If removing the non-compliant dependency is impossible, you will be hung out to dry.
As noted earlier, Scala language designers break compatibility for good technical reasons related to traits. The language is improved and cleaned up with every version, unlike Java which accumulates cruft. In other words, there is a good side to breaking compatibility. Preserving compatibility is an immensely intricate problem with a wide range of consequences. However, it is ultimately a political decision balancing between stability and change.
Once you have tasted the expressive power of Scala, it is hard to go back to program in Java. Once you have tasted the stability of Java, it is hard to put up with the brittleness of Scala. It's a non-ideal world out there.
Tooling proposed by Typesafe detects breakages and ensuring compatibility in minor versions. This tool is similar to clirr which has been around for a long time. Typesafe's response to the binary compatibility issue confirms my suspicions that the issue is still largely misunderstood by Typesafe. Typesafe subscription, the Migration manager or taking over a larger set of core libraries by Typesafe do not ensure that upgrading a project to the next version of Scala will go smoothly.
Assuming Scala continues to break compatibility in the foreseeable future, then I'll go out on a limb and make the following predictions:
The current situation limits the Scala user-base to a relatively small niche of enthusiasts. The small user-base hinders the development of a large Scala eco-system which further limit growth of the user-base, creating a vicious cycle.
Apparently, only few people complain about Scala's existing compatibility policy. Presumably, the Scala community has entered a comfort zone where existing users have grown accustomed to the current situation. For example, SBT makes it easy for authors of Scala libraries to generate artifacts for multiple versions of Scala. However, SBT is not suitable for projects which offer Scala-based extensions but otherwise are centered around Java. Thus, Scala's current compatibility policy makes it hard for Java projects to offer Scala-based extensions. I, for one, would love to offer a Scala-based configurator for logback (in addition to XML and Groovy based configurators) but have no intention of migrating our build to SBT.
One might also forget that the vast vast majority of developers will vote with their feet. They will simply walk away instead of engaging the Scala community for the preservation of binary compatibility. This, Scala will probably continue to be attractive for projects where occasional compatibility breakages are acceptable. Of course, the set of projects where breakages are unacceptable is... non-negligible.
The upcoming Java 8 with support closures will be a big leap forward for the Java platform. Competing languages will eventually close the gap, and Scala will stop being cool.
Friday, August 26, 2011
Saturday, October 30, 2010
Using Groovy and Scala, my first year
On Groovy
After programming almost exclusively in Java for the last 14 years, I picked up Groovy during the spring of 2010. If my memory serves me correctly, it took me about a week. Groovy is a powerful and worthy language. Many things which are hard to do in Java are easy in Groovy. The ExpandoMetaClass makes it very easy to write amazingly powerful DSLs in a jiffy. To give you a concrete example, I could replicate logback's XML-based configuration sub-system of about 10'000 lines of rather intricate code with about 500 lines of Groovy. The groovy-based configuration DSL for logback is better than its XML counterpart in every respect. Not only is the groovy syntax much shorter than XML, it is also internal. This means that the logback configuration DSL preserves all the power of Groovy, a fully-fledged programming language. In short, Groovy is fantastic for writing internal DSLs.Now the bad news. Contrary to Java, Groovy has a dynamic type system which is another way of saying that Groovy has dynamic dispatch. Dynamic-dispatch allows for a lot of flexibility as offered by ExpandoMetaClass and mixins but at the cost of a very significant performance hit. Last but not least, the Groovy compiler skips type checking which leaves you to discover errors at runtime.
A basic for-loop can be 1000 times slower to execute in Groovy than in Java. I don't think it is possible to build performance critical systems using Groovy. But I might be wrong and my groovy-is-slow gripe being as irrelevant as C++ folks complaining about the "slowness" of Java byte code.
By the way, my current IDE of choice is IntelliJ IDEA which has pretty decent support for Groovy as well as Scala. I switched away from Eclipse because of its shoddy support for Groovy and even worse support for Scala.
On Scala
Compared to Groovy, Scala has been harder to learn for me. While Groovy can be learned piecemeal, one has to know a lot of Scala before being able program in Scala. I think this is true for any new language, groovy being a rare-exception since it is essentially a super-set of Java.In exchange for a steeper learning curve, Scala rewards the apprentice with what seems like a fresh perspective on programming: the functional view. Like Java, Scala is statically typed which means that Scala generated code can be as efficient as Java generated code.
As statically-typed language, the Scala compiler catches many programming errors early in the development cycle. However, thanks to its amazingly powerful type-inference system scala dispenses with much the boilerplate that is found in other statically typed languages. Scala also provides strong support for writing internal DSLs. I have written a configuration DSL for logback in Scala. The still unpublished result is as flexible and powerful as its Groovy counterpart and only slightly more verbose. On the other hand, the Scala version is strongly-typed which means that one can write logback configuration files with code-completion provided by the IDE. Is that cool or what?
Scala has a long series of very neat features. I like Scala to the point of animating the Scala Enthusiasts Group in Lausanne, Switzerland. Yes, the Scala requires some effort to learn. Yes, the Scala compiler is slower than most. Yet, those disadvantages are compensated by the consistency and the sheer expressive power of the Scala language.
On the downside, Scala has a consistent history of braking compatibility between versions. For example, code compiled with Scala version 2.7 will not work with Scala version 2.8. Starting with Scala version 2.9, the Migration Manager will purportedly address this binary-format incompatibility problem.
However, from what I understand, Scala suffers from a much hairier compatibility problem. Indeed, Scala traits are copied/folded into code importing the trait. This means that client code compiled with Scala version 2.x.y-revision_A might not be compatible with Scala version 2.x.y_revision_B (with x and y unchanged) if some trait changed between releases.
If this assertion is true, and I pray the God of Evolution that it is false, then
Scala suffers from a fatal flaw. There is no way an ecosystem of libraries written in Scala can prosper when one has to worry about the micro release version of Scala used to compile a given library. Applications using just a couple of libraries written in Scala would be unmanageable. It is pretty ironic that Scala forfeits one of Java's crucial advantages, namely cross-platform compatibility, in exchange for multiple-inheritance as provided by traits.
Again, I may be completely misunderstanding the implications of Scala traits. If you know better about how Scala traits work, you are hereby invited to set the record straight.
Update
According to new information I received, it appears that traits are imported via stubs. Let A be a class importing a trait T. The compiler will create a stub for T in A and invocations of T's methods in A will be delegated to an encapsulated instance of T by the stub. The code in T is not copied into A, only T's method signatures are directly visible in A.It follows that A compiled with one version of Scala and T will be compatible with a different version of Scala as long as T's interface, i.e. method signatures, remains the same (or change in a compatible way) and the Scala language versions are binary compatible.
In other words, traits are not a source of new incompatibility issues as long as the method signatures of a trait are not changed in incompatible ways, much like Java interfaces can only change in certain limited ways in order to preserve compatibility.
QOS.ch, main sponsor of cal10n, logback and slf4j open source projects, is looking to hire talented software developers. If interested, please email your resume to hr@qos.ch.
Tuesday, May 25, 2010
Committocracy as an alternative for conflict resolution in OSS projects
I my previous post I presented an unsparing criticism of Apache's voting procedures. In this post, I describe a new voting procedure where each committer's voting rights derive from the number of commits made by the committer in question. To avoid rewarding micro-commits, only a single commit-point is awarded per day for one or more commits. No points are awarded for a committer who makes no commits on a given day.
Non controversial questions are settled by consensus. However, whenever a decision cannot be reached by unanimous agreement, a vote is called for. The commit-point for or against a motion are summed, the total accumulated commit-points determining the outcome.
Such a system endows a collaborative software project, open source or not, to reach agreement in a timely and orderly fashion. It is much less vulnerable to disruption by unscrupulous participants than Apache's current voting procedures. I designate an organization where voting power derives from the number of commits made by an individual, typically a software developer, as a committocracy.
In the case of git, the following command can be used to compute the commit-points accumulated by Alice.
In a committocracy, voting power accrues with each day involving a commit. As the entitlement of the committer grows with each contribution, the positive psychological effects of belonging to the community may be longer lasting, in particular because accrual in voting rights it is deserved, i.e. based on "merit". However, since I am not aware of any committocracy in existence, I can only speculate on the longer duration of said positive effects.
While a committocracy is not a perfect or even a very good measure of merit, it is still a much better measure than a 0 or 1 switch correlative with veto powers. Since OSS contributors surrender so much of their independence to the OSS organization, in this case the ASF, the organization must ensure that its rules are as just as possible.
Non controversial questions are settled by consensus. However, whenever a decision cannot be reached by unanimous agreement, a vote is called for. The commit-point for or against a motion are summed, the total accumulated commit-points determining the outcome.
Such a system endows a collaborative software project, open source or not, to reach agreement in a timely and orderly fashion. It is much less vulnerable to disruption by unscrupulous participants than Apache's current voting procedures. I designate an organization where voting power derives from the number of commits made by an individual, typically a software developer, as a committocracy.
Feasibility
Given any reasonably-open version control system, it should be fairly easy to write software which assigns commit points to each committer. During a vote it is a matter of simple arithmetic to tally the commit-points expressed for or against a motion.In the case of git, the following command can be used to compute the commit-points accumulated by Alice.
git log --format='%ad %an' --date=short|uniq|grep Alice|wc -l
Worse of both worlds or best of both worlds?
A committocracy is less efficient than the BDFL model for decision making, and compared to the Apache-way, it grants less power to newcomers. However, a committocracy is a fair system in the sense that the same rules apply to all. Today's committer with the most committer-points can be different than that of tomorrow. Moreover, compared to the Apache-way, a committocracy drastically reduces the risk of a project going haywire after admitting a new member. As a corollary, a project can safely reduce the wait-and-see period preceding the admission of new committers. Thus, newcomers may be granted committership status more quickly.Psychological effects
Immediately after electing a new committer, accepting him or her as an equal member of the community has powerful positive effects on the psyche of the electee. In a minority of cases it results in the electee becoming a major contributor of the project. However, this positive effect is often short-lived. After a few months of modest activity, it can even transform itself to sense of entitlement which can be devastating at the hands of a less-than-scrupulous committer wielding veto powers. Fortunately, most people are fairly decent and do not abuse their rights in any blatant way.In a committocracy, voting power accrues with each day involving a commit. As the entitlement of the committer grows with each contribution, the positive psychological effects of belonging to the community may be longer lasting, in particular because accrual in voting rights it is deserved, i.e. based on "merit". However, since I am not aware of any committocracy in existence, I can only speculate on the longer duration of said positive effects.
Is a committocracy a meritocracy?
No, not exactly. Granting one voting point per day rewards participants most committed to the project over a lengthy period. It does not directly take into account the value of each commit. In a very general sense, granting veto rights to all committers can also be considered as a meritocracy because it rewards participants with most endurance in arguments. It is just a different, and in my opinion less pertinent, measure of merit.Does it matter?
What is the point of a committocracy , since it obviously does not perfectly capture the notion of merit? Moreover, why change the current voting system if only a few people complain about it?While a committocracy is not a perfect or even a very good measure of merit, it is still a much better measure than a 0 or 1 switch correlative with veto powers. Since OSS contributors surrender so much of their independence to the OSS organization, in this case the ASF, the organization must ensure that its rules are as just as possible.
Friday, May 21, 2010
The forces and vulnerabilities of the Apache model
The initial title for this article was "Why the Apache model sucks". It would have been a catchier title but would taint my arguments with triviality. But it was the first title that came to my mind and you should be aware of that.
I have written about Apache in the past past and the present post is a rehash with a slightly different emphasis. Before laying further criticism at the altar of the "Apache Way", it must be mentioned that Apache is one of the most open and transparent organizations I know of. Transparency comes at a price. If one is allowed or even encouraged to voice criticism, the voices of the critics may drown the successes of the organization so that the state of affairs may seem bleaker than it is really is.
On the other hand, an organization can be wildly successful even if some of its governing rules prove to be counter-productive on the long run. As a very extreme example, in the Ottoman-empire, one of the most successful and relatively recent empires in history, whenever a new sultan acceded to power he would proceed to kill all his brothers. Not only fratricide was authorized, in the absence of clear succession rules other than the survival of the fittest, the people actually expected the new ruler to kill all his brothers so that the strongest ruler could emerge. The rulers who chose to spare the lives of their brethren were accused of not getting the Ottoman-way. Suleiman the Magnificent went as far as killing his own son Mustafa after bogusly charging him of treason.
One of the the core tenets of Apache is meritocracy. If Apache is a meritocracy, it is a lousy one and this has consequences. If some developer, say David, shows interest in some Apache project, say Prokayotae, and contributes by submitting patches, or by responding to users on the mailing list for a period of time, usually three months or more, then he will be awarded committership. As Prokayotae committer, David can veto decisions of the project – yes, that is veto with a 'v'. After a few years of good behavior, David could even be co-opted to become a member of the Apache Software Foundation. Being a member is a nice honorary title but does not carry further entitlements at the project level. As a member David can indeed vote for the Apache Board during elections but that is mostly inconsequential at the project level.
Coming back to Prokayotae, the newly elected committer David wields the same voting power as Carol, who has been actively contributing to Prokayotae over several years. In most cases David will be a reasonable individual who play nice and will naturally defer to Carol's and other committers' opinions by virtue of common decency and their past involvement in Prokayotae. However, while most new developers are reasonable individuals and make every effort to play nice, some people are less reasonable. Even a modestly successful project will have over a dozen committers elected over its lifetime. Thus, the probability of electing an "unreasonable" individual increases over time, and I'd venture to say, approaches certainty.
Let Ursula be such an unreasonable person elected as a Prokayotae committer. Ursula maybe otherwise a nice person overall and a valuable member of the community she lives in. But, as a fervent believer in Thor, one night Ursula has a dream where Thor ordains her to oppose the pernicious influence of Carol over the Prokayotae project. Of course, the motivations for Ursula's opposition to Carol may have origins other than Thor appearing to her in a dream. The potential reasons are innumerable. It may be as simple as a design principle lambda that Ursula reveres but which Carol reveres less.
Invested with this new mission in life, Ursula begins to challenge Carol by vetoing her commits or merely intimating that she will veto them based on the lofty lambda design principle. Were Ursula's objections patently silly, she will be revealed as a fool. For example, invoking Thor's appearing in her dreams as a justification for her opposition will expose her to ridicule. However, any reason which is based on some apparently reasonable pretext, often adherence to a lofty principle, provides sufficient cover for long-term disruption. Every developer knows that the IT industry can claim more than its fair share of high and often contradictory principles. Ursula will make convoluted arguments, misrepresent facts and argue endlessly. Given that each new commit takes ages in endless arguments, further development in Prokayotae will be severely disrupted and may even cease.
Carol who has invested heavily in Prokayotae and got along well with the other developers may be shocked and ill-prepared to deal with Ursula's disruptive interventions. Carol may be further shocked to see other committers sitting on the fence with respect to Ursala's arguments cloaked under an important design principle such as lambda. At this stage, Carol must tread with utmost care because she formally has the same voting rights as any other committer, including Ursula. If Carol dares to claim that her arguments should carry more weight in light of the volume of her past contributions, she will immediately lose her credibility.
The Apache culture aggressively enforces its egalitarian model. Carol, even if she effectively led the project for several years, is not allowed the title of project leader or project manager. Author tags in source code are also frowned upon with the Apache board having formulated a statement discouraging their use. Hints at unequal merit are met with condescension and/or alarm.
As a result, if Carol tries to explain the injustices of the Apache way she will be branded as clueless and subsequently ignored. Being ignored is Apache's equivalent to being ostracized in the pre-Internet age.
As most people who face injustice and have a choice, Carol will leave Prokayotae as I have left log4j to start the SLF4J and logback projects. Leaving log4j was one of the most traumatic experiences of my life. Trustin Lee, of Netty and Mina fame, had an analogous experience. He apparently could not bear to see his work being vetoed over a triviality.
Those favoring the current voting system, while recognizing its lopsidedness, argue that letting everyone having an equal voice fosters communication between developers. Admittedly, the current set up is conducive to collaboration as progress can only be achieved after reaching consensus. As a corollary to this line of reasoning, letting all arguments be heard and all objections raised at every step of the development process must surely lead to best possible software.
Unfortunately, in the absence of a fair conflict resolution mechanism, having lots of ideas floating around emanating from different people does not lead to the emergence of the best software design insofar as it promotes bickering and political paralysis. In such a system where development can be easily disrupted, it is not the best ideas that win but the ideas advocated by those possessing the most stamina.
Instead of trying to learn from past failures which open discussion is supposed to encourage, Apache forges on in the path of egalitarianism. As time passes, I see attempts at institutionalizing egalitarianism instead of recognizing its inherent injustice. For the sake of a bogus ideal, the Apache-way expects selflessness on the part of the developer in the same way that the catholic church expects celibacy from its priests. If egalitarianism is really at the core of the Apache way as an absolute value, then the Apache way sucks. Yay!
While the one person one vote principle applies to a democracy in order to run a country for the benefit of all, the one committer one veto principle is ill-suited in a purported meritocracy such as Apache. If it must be "one committer one veto", then the word meritocracy cannot be honestly ascribed to Apache. It it would be most appropriate for the ASF to stop misrepresenting itself as a meritocracy, at least not without clearly defining the meaning of merit.
The Apache culture fails to recognize that project participants may have drastically different investments in a project. Allowing a person with near-zero involvement in a project to hold the same weight as a person with 10'000 hours of investment is to date an essential part of the Apache-way. According to the prevailing Apache culture, if you don't completely agree with this premise, body and soul, then you just don't get it.
I hope that this blog entry will incite OSS organizations, including the ASF, to adopt fairer decision making procedures. If that does not happen, before joining such an organization, developers should at least be aware of the inconveniences associated with lack of fair and orderly decision making mechanisms at the project level. If knowing these inconveniences, a developer decides to join anyway, then it will be the result of an informed decision.
I have written about Apache in the past past and the present post is a rehash with a slightly different emphasis. Before laying further criticism at the altar of the "Apache Way", it must be mentioned that Apache is one of the most open and transparent organizations I know of. Transparency comes at a price. If one is allowed or even encouraged to voice criticism, the voices of the critics may drown the successes of the organization so that the state of affairs may seem bleaker than it is really is.
On the other hand, an organization can be wildly successful even if some of its governing rules prove to be counter-productive on the long run. As a very extreme example, in the Ottoman-empire, one of the most successful and relatively recent empires in history, whenever a new sultan acceded to power he would proceed to kill all his brothers. Not only fratricide was authorized, in the absence of clear succession rules other than the survival of the fittest, the people actually expected the new ruler to kill all his brothers so that the strongest ruler could emerge. The rulers who chose to spare the lives of their brethren were accused of not getting the Ottoman-way. Suleiman the Magnificent went as far as killing his own son Mustafa after bogusly charging him of treason.
One of the the core tenets of Apache is meritocracy. If Apache is a meritocracy, it is a lousy one and this has consequences. If some developer, say David, shows interest in some Apache project, say Prokayotae, and contributes by submitting patches, or by responding to users on the mailing list for a period of time, usually three months or more, then he will be awarded committership. As Prokayotae committer, David can veto decisions of the project – yes, that is veto with a 'v'. After a few years of good behavior, David could even be co-opted to become a member of the Apache Software Foundation. Being a member is a nice honorary title but does not carry further entitlements at the project level. As a member David can indeed vote for the Apache Board during elections but that is mostly inconsequential at the project level.
Coming back to Prokayotae, the newly elected committer David wields the same voting power as Carol, who has been actively contributing to Prokayotae over several years. In most cases David will be a reasonable individual who play nice and will naturally defer to Carol's and other committers' opinions by virtue of common decency and their past involvement in Prokayotae. However, while most new developers are reasonable individuals and make every effort to play nice, some people are less reasonable. Even a modestly successful project will have over a dozen committers elected over its lifetime. Thus, the probability of electing an "unreasonable" individual increases over time, and I'd venture to say, approaches certainty.
Let Ursula be such an unreasonable person elected as a Prokayotae committer. Ursula maybe otherwise a nice person overall and a valuable member of the community she lives in. But, as a fervent believer in Thor, one night Ursula has a dream where Thor ordains her to oppose the pernicious influence of Carol over the Prokayotae project. Of course, the motivations for Ursula's opposition to Carol may have origins other than Thor appearing to her in a dream. The potential reasons are innumerable. It may be as simple as a design principle lambda that Ursula reveres but which Carol reveres less.
Invested with this new mission in life, Ursula begins to challenge Carol by vetoing her commits or merely intimating that she will veto them based on the lofty lambda design principle. Were Ursula's objections patently silly, she will be revealed as a fool. For example, invoking Thor's appearing in her dreams as a justification for her opposition will expose her to ridicule. However, any reason which is based on some apparently reasonable pretext, often adherence to a lofty principle, provides sufficient cover for long-term disruption. Every developer knows that the IT industry can claim more than its fair share of high and often contradictory principles. Ursula will make convoluted arguments, misrepresent facts and argue endlessly. Given that each new commit takes ages in endless arguments, further development in Prokayotae will be severely disrupted and may even cease.
Carol who has invested heavily in Prokayotae and got along well with the other developers may be shocked and ill-prepared to deal with Ursula's disruptive interventions. Carol may be further shocked to see other committers sitting on the fence with respect to Ursala's arguments cloaked under an important design principle such as lambda. At this stage, Carol must tread with utmost care because she formally has the same voting rights as any other committer, including Ursula. If Carol dares to claim that her arguments should carry more weight in light of the volume of her past contributions, she will immediately lose her credibility.
The Apache culture aggressively enforces its egalitarian model. Carol, even if she effectively led the project for several years, is not allowed the title of project leader or project manager. Author tags in source code are also frowned upon with the Apache board having formulated a statement discouraging their use. Hints at unequal merit are met with condescension and/or alarm.
As a result, if Carol tries to explain the injustices of the Apache way she will be branded as clueless and subsequently ignored. Being ignored is Apache's equivalent to being ostracized in the pre-Internet age.
As most people who face injustice and have a choice, Carol will leave Prokayotae as I have left log4j to start the SLF4J and logback projects. Leaving log4j was one of the most traumatic experiences of my life. Trustin Lee, of Netty and Mina fame, had an analogous experience. He apparently could not bear to see his work being vetoed over a triviality.
Those favoring the current voting system, while recognizing its lopsidedness, argue that letting everyone having an equal voice fosters communication between developers. Admittedly, the current set up is conducive to collaboration as progress can only be achieved after reaching consensus. As a corollary to this line of reasoning, letting all arguments be heard and all objections raised at every step of the development process must surely lead to best possible software.
Unfortunately, in the absence of a fair conflict resolution mechanism, having lots of ideas floating around emanating from different people does not lead to the emergence of the best software design insofar as it promotes bickering and political paralysis. In such a system where development can be easily disrupted, it is not the best ideas that win but the ideas advocated by those possessing the most stamina.
Instead of trying to learn from past failures which open discussion is supposed to encourage, Apache forges on in the path of egalitarianism. As time passes, I see attempts at institutionalizing egalitarianism instead of recognizing its inherent injustice. For the sake of a bogus ideal, the Apache-way expects selflessness on the part of the developer in the same way that the catholic church expects celibacy from its priests. If egalitarianism is really at the core of the Apache way as an absolute value, then the Apache way sucks. Yay!
While the one person one vote principle applies to a democracy in order to run a country for the benefit of all, the one committer one veto principle is ill-suited in a purported meritocracy such as Apache. If it must be "one committer one veto", then the word meritocracy cannot be honestly ascribed to Apache. It it would be most appropriate for the ASF to stop misrepresenting itself as a meritocracy, at least not without clearly defining the meaning of merit.
The Apache culture fails to recognize that project participants may have drastically different investments in a project. Allowing a person with near-zero involvement in a project to hold the same weight as a person with 10'000 hours of investment is to date an essential part of the Apache-way. According to the prevailing Apache culture, if you don't completely agree with this premise, body and soul, then you just don't get it.
I hope that this blog entry will incite OSS organizations, including the ASF, to adopt fairer decision making procedures. If that does not happen, before joining such an organization, developers should at least be aware of the inconveniences associated with lack of fair and orderly decision making mechanisms at the project level. If knowing these inconveniences, a developer decides to join anyway, then it will be the result of an informed decision.
Thursday, May 06, 2010
Encouters in Groovy I
You can take any .java file and rename it .groovy and the result will be valid Groovy. Compared to other contenders in the new-Java space such as Scala or Clojure, the syntactical-backward compatibility offered by Groovy is undoubtedly an important and possibly a decisive advantage.
Let me demonstrate. Here is a very simple unit test measuring the performance of a trivial arithmetic operation.
Running the above test yields:
As I don't work for a sensation-driven news organization nor have an anti-Groovy agenda, I will try to mitigate the preceding results.
The code generated by Groovy works on objects instead of primitive types. For example, the 'i < LEN' check is done by invoking the
the
We can actually improve the performance of the loop by using more idiomatic groovy. Modifying the iteration from
We are still far from the 3 nanoseconds obtained from initial .java version of the code. Perhaps the code can be further optimized and close the gap with the original .java version.
Groovy is indeed slower than Java in tight loops. However, it so happens that the performance of most applications is I/O bound, so the practical performance impact of Groovy may be largely offset by the (developer) productivity gains it offers.
In conclusion, while blindly converting .java files to .groovy may result in a catastrophic degradation in performance, a more selective migration can result in significantly better code without serious degradation in performance.
Let me demonstrate. Here is a very simple unit test measuring the performance of a trivial arithmetic operation.
// file MyTest.java
package ch.qos;
import org.junit.Test;
public class MyTest {
static int LEN = 100*1000;
@Test
public void smoke() {
// let the JVM warm up
loop();
loop();
double result = loop();
System.out.println("Average duration per operation: "+result+ " nanoseconds");
}
double loop() {
long start = System.nanoTime();
double sum = 0;
for (int i = 0; i < LEN; i++) {
sum += i*1.0;
}
long end = System.nanoTime();
return (end - start) / LEN;
}
}
Running the above test yields:
Average duration per operation: 3.0 nanosecondsAs mentioned earlier, a valid Java class is also a valid Groovy class. So renaming "MyTest.java" as "MyTest.groovy" results in a valid Groovy class. With JetBrains IDEA which provides pretty nice Groovy support, I can run "MyTest.groovy" as any other junit test. Here is the result:
Average duration per operation: 843.09 nanosecondsLo and behold, the same code runs 280 times slower when compiled as a Groovy class than its Java counterpart. If I were blogging for a sensation-driven news organization with an anti-Groovy agenda, I would now prematurely claim the death of Groovy and stop writing.
As I don't work for a sensation-driven news organization nor have an anti-Groovy agenda, I will try to mitigate the preceding results.
The code generated by Groovy works on objects instead of primitive types. For example, the 'i < LEN' check is done by invoking the
compareLessThan() method in the the
ScriptBytecodeAdapter class part of the groovy runtime. This method operates on objects instead of the primitives types. I suspect that the dynamic-nature of Groovy forces it to invoke methods flexible enough to deal with untyped objects, instead of invoking more trivial byte code which the hot-spot compiler is pretty masterful at optimizing -- but that's just my hunch.We can actually improve the performance of the loop by using more idiomatic groovy. Modifying the iteration from
for (int i = 0; i < LEN; i++) {
sum += i*1.0;
}tofor (i in 1..LEN;) {
sum += i*1.0;
} brings down the average duration per operation from 843 to 675 nanoseconds. By avoiding the integer to double conversions we can drastically improve performance. Here is the modified iteration:for (double i in 1..LEN;) {
sum += i;
} Surprisingly enough, this last optimization brings the average duration to 50 nanoseconds, a 17 fold improvement from the initial non-idiomatic version of the code running at 843 nanoseconds per operation.We are still far from the 3 nanoseconds obtained from initial .java version of the code. Perhaps the code can be further optimized and close the gap with the original .java version.
Groovy is indeed slower than Java in tight loops. However, it so happens that the performance of most applications is I/O bound, so the practical performance impact of Groovy may be largely offset by the (developer) productivity gains it offers.
In conclusion, while blindly converting .java files to .groovy may result in a catastrophic degradation in performance, a more selective migration can result in significantly better code without serious degradation in performance.
Monday, April 05, 2010
Hosni Mobarak and bottled water
Egyptian President Hosni Mubarak was recently operated in Germany to have his gall bladder removed. Hosni Mubarak assumed presidency on October 14th 1981 after the assassination of President Anwar el-Sadat. At the time, many had predicted that Mubarak's rule would not last and he'd be ousted in no time. That was almost 30 years and Mubarak is still the President of Egypt. Of course, Mubarak was smart enough to avoid the cardinal mistake made by Sadat. Contrary to Sadat who surprised everyone by visiting Israel and addressing the Knesset on November 20th 1977
Hosni Mobarak has never set foot on Israeli soil - not once in 30 years. There's a man with lofty principles. But I am digressing.
Why am I picking on a sick old man having surgery in a German hospital and what does it have to do with bottled water? Well, during his 30 year reign, Mubarak could have invested in Egyptian hospitals so that when the day came, he would not need to fly to a a foreign land to have his gall bladder removed. The ruling elite of any country should not be shielded away from the conditions that apply to the rest of the population. Rulers and legislators should have access to the same medical care as everyone else, not better and not worse. Thus, when the day comes to vote the budget of the health ministry, given that state of public hospitals directly affects their own lives, legislators will vote in a way ensuring reasonable medical standards for all, including themselves. There should be a law preventing officials who have serve in high public office to seek medical care in foreign countries.
What does this have to do with bottled water? In many countries, the quality of tap water is better than that of bottled water, for example in Switzerland where Nestlé is headquartered. This hasn't prevented Nestlé's from making outrageous claims about bottled water being better for the environment than tap water. (A few billion dollars is reason enough to bend the truth.) Notwithstanding Nestlé's assertions, it goes without saying that the ecological impact of conditioning and transporting water in individual bottles is orders of magnitude higher than that of tap water.
In other countries such as Turkey where water is still plentiful, the quality of tap water is noticeably worse than bottled water. Instead of making the necessary investments in water treatment installations and water conduits, Turkey's population as well as its politicians are busy drinking bottled water causing further degradation to the environment. Therein lies the danger of bottled water. It gives the illusion that one can be isolated from the surrounding water and air pollution and live healthy lives. In the same way politicians of developing countries attempt to shield themselves from the living conditions of the masses resulting in the continued neglect of public hospitals, bottled water allows the masses to isolate themselves from their environment. On the long run, this strategy is as efficient as sticking one's head in the sand which admittedly provides momentary relief.
Hosni Mobarak has never set foot on Israeli soil - not once in 30 years. There's a man with lofty principles. But I am digressing.
Why am I picking on a sick old man having surgery in a German hospital and what does it have to do with bottled water? Well, during his 30 year reign, Mubarak could have invested in Egyptian hospitals so that when the day came, he would not need to fly to a a foreign land to have his gall bladder removed. The ruling elite of any country should not be shielded away from the conditions that apply to the rest of the population. Rulers and legislators should have access to the same medical care as everyone else, not better and not worse. Thus, when the day comes to vote the budget of the health ministry, given that state of public hospitals directly affects their own lives, legislators will vote in a way ensuring reasonable medical standards for all, including themselves. There should be a law preventing officials who have serve in high public office to seek medical care in foreign countries.
What does this have to do with bottled water? In many countries, the quality of tap water is better than that of bottled water, for example in Switzerland where Nestlé is headquartered. This hasn't prevented Nestlé's from making outrageous claims about bottled water being better for the environment than tap water. (A few billion dollars is reason enough to bend the truth.) Notwithstanding Nestlé's assertions, it goes without saying that the ecological impact of conditioning and transporting water in individual bottles is orders of magnitude higher than that of tap water.
In other countries such as Turkey where water is still plentiful, the quality of tap water is noticeably worse than bottled water. Instead of making the necessary investments in water treatment installations and water conduits, Turkey's population as well as its politicians are busy drinking bottled water causing further degradation to the environment. Therein lies the danger of bottled water. It gives the illusion that one can be isolated from the surrounding water and air pollution and live healthy lives. In the same way politicians of developing countries attempt to shield themselves from the living conditions of the masses resulting in the continued neglect of public hospitals, bottled water allows the masses to isolate themselves from their environment. On the long run, this strategy is as efficient as sticking one's head in the sand which admittedly provides momentary relief.
Wednesday, March 31, 2010
A middle-class man's conditional logging configuration
In an earlier post I described how one adapt a single configuration file for use in development as well as production environments. At those prehistoric times, that is January 2010, logback did not support conditional (if-then-else) configuration statements. So we made use of a default substitution value to do the the trick.
However, as of version 0.9.20 (to be released early April), logback supports conditional configuration statements. Thus, we can write:
In this particular case, the if-then-else approach is more verbose than default variable substitution, i.e. simply writing
However, as of version 0.9.20 (to be released early April), logback supports conditional configuration statements. Thus, we can write:
<appender name="FILE" class="ch.qos.logback.core.FileAppender">Thus, if the "catalina.home" system property is defined (non null), then the log file will be written to ${catalina.home}/logs/myApp.log, otherwise, we are in the development environment and the logs should be written to ./logs/myApp.log.
<if condition='!isNull("catalina.home")'>
<then>
<File>${catalina.home}/logs/myApp.log</File>
</then>
<else>
<File>./logs/myApp.log</File>
</else>
</if>
...
</appender>
In this particular case, the if-then-else approach is more verbose than default variable substitution, i.e. simply writing
${catalina.home:-.}. However, it goes without saying that conditional configuration brings much needed flexibility to the table. Keep in mind that given the verbosity of XML, one could gradually end up with a configuration file containing many logical branches rendering the file ungrokable. In case you ask, nested if-then-else statements are supported. So please use conditionals with moderation so that complexity does not creep into your configuration files.
Sunday, March 28, 2010
Dealing with ignorable errors
In real-world applications assembled out of heterogeneous parts or interacting with systems beyond your control, error conditions are bound to occur. Often times, some of these errors can be safely ignored. For example, the qos.ch e-commerce site is crawled by googlebot which for some unknown reason insists on visiting invalid URLs about 30-50 times a day. These invalid requests cause the wicket server to throw exceptions of type
Given that Wicket relies on SLF4J for logging, the
Below is a configuration file which instructs
It relies heavily on
You could also throw in markers into the mix as described in a previous post. It should be noted that the googlebot problem should be addressed by setting an internal error page but that's a different story.
WicketRuntimeException.Given that Wicket relies on SLF4J for logging, the
WicketRuntimeException is logged as an error. To keep abreast of errors in our e-commerce applications, we use SMTPAppender in our logging configuration. Thus, every event logged as an error triggers an email to our site's administrator. As you can imagine, receiving over 30 emails per day as a result of erroneous Googlebot submissions defeats the purpose of these log report sent by email. (After the 100th bogus email, the admin stops caring.)Below is a configuration file which instructs
SMTPAppedner to ignore errors generated by Googlebot. <configuration scan="true">
<statusListener class="ch.qos.logback.core.status.OnConsoleStatusListener" />
<appender name="EMAIL" class="ch.qos.logback.classic.net.SMTPAppender">
<layout class="ch.qos.logback.classic.html.HTMLLayout">
<pattern>
%date%-5level%thread%X{req.remoteHost}%logger%msg
</pattern>
</layout>
<From>...</From>
<SMTPHost>...</SMTPHost>
<Subject>[${HOSTNAME}] %msg</Subject>
<To>...</To>
<evaluator class="ch.qos.logback.classic.boolex.JaninoEventEvaluator">
<expression>
level>=WARN
&&
!( mdc.get("req.userAgent") != null && ((String) mdc.get("req.userAgent")).contains("Googlebot"))
</expression>
</evaluator>
</appender>
<root level="INFO">
<appender-ref ref="EMAIL" />
</root>
</configuration>
It relies heavily on
JaninoEventEvaluator and requires the installation of MDCInsertingServletFilter.You could also throw in markers into the mix as described in a previous post. It should be noted that the googlebot problem should be addressed by setting an internal error page but that's a different story.
Friday, January 22, 2010
A poor man's conditional logging configuration
It is often desirable to keep the same logging configuration file during development and production environments. (I use logback as my logging framework.)
Ideally, one would check whether the application was running on the production environment and enable parts of the configuration file accordingly. Unfortunately, logback does not support conditional (if-then-else) configuration statements.
However, it supports default substitution for variables which can be used to solve a variety of problems. For example, I would like to have a file FileAppender which logs under the logs/ folder of my Tomcat server. Here is the relevant configuration snippet:
./logs/myApp.log within my development environment. Easy as pie.
Ideally, one would check whether the application was running on the production environment and enable parts of the configuration file accordingly. Unfortunately, logback does not support conditional (if-then-else) configuration statements.
However, it supports default substitution for variables which can be used to solve a variety of problems. For example, I would like to have a file FileAppender which logs under the logs/ folder of my Tomcat server. Here is the relevant configuration snippet:
<appender name="FILE" class="ch.qos.logback.core.FileAppender">As long as the catalina.home property is defined, the above works as expected. However, during development I use the jetty server which does not define the said property. I can still instruct logback to fallback to a reasonable value by specifying a default substitution value for ${catalina.home}. The current directory, symbolized by a dot, i.e. '.', is quite a reasonable default. The configuration snipped is thus modified as:
<File>${catalina.home}/logs/myApp.log</File>
...
</appender>
<appender name="FILE" class="ch.qos.logback.core.FileAppender">This will have the log file of my application write to
<File>${catalina.home:-.}/logs/myApp.log</File>
...
</appender>
./logs/myApp.log within my development environment. Easy as pie.
301 redirects with Wicket
Sometimes a web-page needs to redirect to a new address. A 301 redirect means that a page moved permanently, which in most cases is indistinguishable from a 302 "moved temporarily" redirect. Search engines however react differently to a 301 response than to a 302 response. If a page has moved permanently, sending a 301 will cause the search engine to update its index to the new page. For "important" pages, you would thus prefer sending 301 redirect from an old url to a new url.
Assuming the old url was a .jsp, here is a JSP snippet to send a 301 redirect:
If you are using Wicket, you can instruct the wicket filter in your web.xml file to ignore the said url, otherwise wicket filter would try to serve the JSP page as a Wicket page, which would not work.
Assuming the old url was a .jsp, here is a JSP snippet to send a 301 redirect:
<%
String redirectURL = "http://.../";
response.setStatus(301);
response.setHeader( "Location", redirectURL);
response.setHeader( "Connection", "close" );
%>
If you are using Wicket, you can instruct the wicket filter in your web.xml file to ignore the said url, otherwise wicket filter would try to serve the JSP page as a Wicket page, which would not work.
<filter>For an arbitrary (non-jspP) url, you might actually need to create a wicket page redirecting from the old url to the new one. Here is the relevant code:
<filter-name>a name</filter-name>
<filter-class>org.apache.wicket.protocol.http.WicketFilter</filter-class>
<init-param>
<param-name>ignorePaths</param-name>
<param-value>oldURL.jsp,other_urls_to_ignore,...</param-value>
</init-param>
</filter>
public class MyRedirectingPage extends WebPage {
public MyRedirectingPage() {
final String newURL = "http://...";
RedirectRequestTarget target = new RedirectRequestTarget(redirectURL) {
@Override
public void respond(RequestCycle requestCycle) {
WebResponse response = (WebResponse) requestCycle.getResponse();
HttpServletResponse servletResponse = response.getHttpServletResponse();
servletResponse.setStatus(HttpServletResponse.SC_MOVED_PERMANENTLY);
servletResponse.setHeader("Location", newURL);
servletResponse.setHeader("Connection", "close");
}
};
getRequestCycle().setRequestTarget(target);
}
}Within your WebApplication class, you also need to mount to the old url. mountBookmarkablePage("path/to/old/url", MyRedirectingPage.class);
Wednesday, January 20, 2010
Marker based email notifications
Part of the SLF4J API, Markers allow developers to add meta-data to log statements. Not all errors are equal, with some errors being more unusual than others. If you wish to color log statement related to "unusual" errors you could write:
whereas a "regular" error might be logged as
So what do markers give you? Well, for one, you can perform marker-based filtering. For example, turn off logging except for errors marked as UNUSUAL. At QOS.ch, we use markers to trigger outgoing emails for certain types of unusual errors.
Here is the configuration snippet:
This way whenever an "unusual" error occurs, the admin is notified via email. The definition of unusual is highly dependent on the application, but the SLF4J and logback combo offers a flexible way of defining and processing "unusualness".
Marker unusualMarker = MarkerFactory.getMarker("UNUSUAL");
Logger logger = LoggerFactory.getLogger("myPackage.myClass");
logger.error(unusualMarker, "An unusual error occurred.");whereas a "regular" error might be logged as
Logger logger = LoggerFactory.getLogger("myPackage.myClass");
logger.error("An error occurred.");So what do markers give you? Well, for one, you can perform marker-based filtering. For example, turn off logging except for errors marked as UNUSUAL. At QOS.ch, we use markers to trigger outgoing emails for certain types of unusual errors.
Here is the configuration snippet:
<appender name="EMAIL" class="ch.qos.logback.classic.net.SMTPAppender">
<layout class="ch.qos.logback.classic.html.HTMLLayout">
<pattern>%date%-5level%thread%mdc{user-id}%logger%msg</pattern>
<throwableRenderer class="ch.qos.logback.classic.html.DefaultThrowableRenderer" />
</layout>
<From>...</From>
<SMTPHost>...</SMTPHost>
<Subject>%msg</Subject>
<To>email address of @ the admin</To>
<evaluator class="ch.qos.logback.classic.boolex.OnMarkerEvaluator">
<marker>UNUSUAL</marker>
<marker>ANOTHER_UNUSUAL_EVENT</marker>
</evaluator>
</appender>
This way whenever an "unusual" error occurs, the admin is notified via email. The definition of unusual is highly dependent on the application, but the SLF4J and logback combo offers a flexible way of defining and processing "unusualness".
Wednesday, December 30, 2009
Not allowed to stand up before landing
According to new flight safety regulations, passengers are not allowed to stand up and go to the restroom one hour prior to landing of the aircraft. How about handcuffing all passengers and tie them to their seats so that they do not budge? Better yet, how about restricting air travel on a need-to-fly basis? Surely, that would improve security...
The USA, as a civilized western democracy, should reign in bureaucrats in charge of flight safety regulations. The current mindless race for safety measures is hopelessly ineffective and is wasting millions of hours of passengers' time. Worse, it signals to those who wish to harm our societies that America is in a state of panic.
Terrorist no longer even have to mount a successful attack. By disrupting airline travel to its present extent, they are already causing serious damage to us all.
Given that it is not possible to physically prevent all terrorist attacks, instead of imposing increasingly stringent measures on the general population, western-civilization should mount a sustained cultural response to counter the retrograde mentality that breeds extremism. A TV-station in Arabic, broadcasting from the Maghreb to Yemen, could present a gentler and more caring face of our civilization. At the another level, local teachers could be helped in various ways to teach their pupils that there is more to life than blowing oneself up in a "martyrdom operation" in the hope of getting chicks in paradise. If the candidates for "martyrdom operations" were taught that using one's life as well as that of others', regardless of creed, as a means to a political-end, was unjust, they would refuse to participate in such operations. If in addition, the candidates were both unwilling and critical, those commissioning the attacks would presumably be hesitant as well.
We need to engage the Muslim world, not by succumbing to its purported demands, nor by militarily invasion, but through multiple channels and in ways respecting the dignity of each. Countering Salafist ideology is too important to be left to a minority of journalists such as Caroline Fourest and Mohamed Sifaoui.
The USA, as a civilized western democracy, should reign in bureaucrats in charge of flight safety regulations. The current mindless race for safety measures is hopelessly ineffective and is wasting millions of hours of passengers' time. Worse, it signals to those who wish to harm our societies that America is in a state of panic.
Terrorist no longer even have to mount a successful attack. By disrupting airline travel to its present extent, they are already causing serious damage to us all.
Given that it is not possible to physically prevent all terrorist attacks, instead of imposing increasingly stringent measures on the general population, western-civilization should mount a sustained cultural response to counter the retrograde mentality that breeds extremism. A TV-station in Arabic, broadcasting from the Maghreb to Yemen, could present a gentler and more caring face of our civilization. At the another level, local teachers could be helped in various ways to teach their pupils that there is more to life than blowing oneself up in a "martyrdom operation" in the hope of getting chicks in paradise. If the candidates for "martyrdom operations" were taught that using one's life as well as that of others', regardless of creed, as a means to a political-end, was unjust, they would refuse to participate in such operations. If in addition, the candidates were both unwilling and critical, those commissioning the attacks would presumably be hesitant as well.
We need to engage the Muslim world, not by succumbing to its purported demands, nor by militarily invasion, but through multiple channels and in ways respecting the dignity of each. Countering Salafist ideology is too important to be left to a minority of journalists such as Caroline Fourest and Mohamed Sifaoui.
Thursday, December 17, 2009
Eclipse template for creating new loggers
Here is the Eclipse template I use for creating new logger fields in my code.
The import and the logger declaration are on the same line, i.e. without a separating new line character, whereas ${cursor} is placed on a new line.
To insert the template in Eclipse, go to Window->Preferences->Java->Editor->Templates and select "New...". I usually name the template as "nlog".
Once the template is inserted, you can insert a new logger in Java code, by typing "nlog" (the name of the template) followed by Ctrl-Space.
HTH,
${:import(org.slf4j.Logger, org.slf4j.LoggerFactory)}Logger logger = LoggerFactory.getLogger(${enclosing_type}.class);
${cursor}
The import and the logger declaration are on the same line, i.e. without a separating new line character, whereas ${cursor} is placed on a new line.
To insert the template in Eclipse, go to Window->Preferences->Java->Editor->Templates and select "New...". I usually name the template as "nlog".
Once the template is inserted, you can insert a new logger in Java code, by typing "nlog" (the name of the template) followed by Ctrl-Space.
HTH,
Friday, November 27, 2009
Just got my Rhino from Emperor Linux
After about 6 years of hesitation, I got a new laptop. The old one was getting embarrassingly slow. Given my past horrible experiences with Dell's customer service, I had decided to buy from another manufacturer. Then, I stumbled upon EmperorLinux which delivers Linux based laptops on top of laptops from Lenovo, Sony and Dell. When contacted, the salesperson at EmperorLinux seemed very knowledgeable and helpful, so instead of buying from Dell, I ordered my new laptop from them. It is a Dell Latitude E6550 rebranded as Rhino Linux.
In addition to a glitch on the graphics card, the laptop weighs 17% more than expected. Dell advertises the Latitude E6550 at 2.3 kg with a 4 cell battery and without an optical reader. According to the user manual, the 4 cell battery weighs .24 kg. The laptop actually delivered weighs 2.7kg, albeit with a 6 cell battery and without the optical reader, or 17% more than advertised. One might argue that the 6 cell battery weights more than a 4 cell battery. Well, the 6 cell battery weighs 325g, or just 90g more than the 4 cell one. After removing the battery, the laptop weighs 2.375 kg. In other words, the laptop actually delivered with its battery and optical reader removed, still weighs more than what is advertised by Dell (2.3kg) for the given laptop with a 4 cell battery.
So, either,
1) Dell builds their laptops on a planet with a different gravitational pull,
2) the laptop will loose weight after some exercise,
3) Dell makes false declarations about the weight of their products.
In addition to a glitch on the graphics card, the laptop weighs 17% more than expected. Dell advertises the Latitude E6550 at 2.3 kg with a 4 cell battery and without an optical reader. According to the user manual, the 4 cell battery weighs .24 kg. The laptop actually delivered weighs 2.7kg, albeit with a 6 cell battery and without the optical reader, or 17% more than advertised. One might argue that the 6 cell battery weights more than a 4 cell battery. Well, the 6 cell battery weighs 325g, or just 90g more than the 4 cell one. After removing the battery, the laptop weighs 2.375 kg. In other words, the laptop actually delivered with its battery and optical reader removed, still weighs more than what is advertised by Dell (2.3kg) for the given laptop with a 4 cell battery.
So, either,
1) Dell builds their laptops on a planet with a different gravitational pull,
2) the laptop will loose weight after some exercise,
3) Dell makes false declarations about the weight of their products.
Friday, October 23, 2009
Page links with Wicket
The default urls created by Wicket do not look very nice. For instance, let IndexPage be the home page of our ShopApplication. Assuming
Wicket has a mechanism for prettifying URLs. In ShopApplication, you can instruct wicket to "mount" a page on a specific path. Here is the code.
Now, assume you have a navigation panel, included on all the pages of your application, where you list links to the various pages of your application. Here is the markup:
For bookmarkable links such as "index", Wicket offers the
link components for the links inside the tags. So let's try that.
Indeed, the
So, we seem to be stuck. Googling for "wicket url mountBookmarkablePage" you will find an article by R.J. Lorimer on controlling Wicket URLs which did not go far in reducing my level of confusion. After further investigation, I found an example which caters for the use case we've been discussing so far, that is the nice-url example.
The nice-url example, after mounting the
It does not know. Instead, it invokes the
shop is your application's context name and wicket the path where you mapped the Wicket filter, then by default, the url for the IndexPage will be akin tohttp://somehost/shop/wicket/index?wicket:interface=:0:headerPanel:index2::ILinkListener::. I said ugly didn't I?Wicket has a mechanism for prettifying URLs. In ShopApplication, you can instruct wicket to "mount" a page on a specific path. Here is the code.
public class ShopApplication extends WebApplication {
@Override
public Class getHomePage() {
return IndexPage.class;
}
@Override
protected void init() {
mountBookmarkablePage("index", IndexPage.class);
}
} Wow, that was easy! Now, assume you have a navigation panel, included on all the pages of your application, where you list links to the various pages of your application. Here is the markup:
<table cellpadding="0" cellspacing="0">We are referencing the mounted path "index" directly. How convenient is that? Although, convenient such references do not work correctly. The navigator will translate it into
<tr align="center">
<td><a class="menuitem" href="index">Index</a></td>
... other pages
</tr>
</table>
http://somehost/shop/index instead of the correct http://somehost/shop/wicket/index. For bookmarkable links such as "index", Wicket offers the
<wicket:link> tag instructing Wicket to to automatically create bookmarkablelink components for the links inside the tags. So let's try that.
<tr align="center">The above is easy enough and will work, as long as all your pages are in the same java package.
<wicket:link>
<td><a class="menuitem" href="index">Index</a></td>
... other pages
</wicket:link>
</tr>
Indeed, the
<wicket:link> tag doctors url references with respect to the java page of the current page belongs to, but if the page is located in a different package, then the urls will be rendered incorrectly. Bummer.So, we seem to be stuck. Googling for "wicket url mountBookmarkablePage" you will find an article by R.J. Lorimer on controlling Wicket URLs which did not go far in reducing my level of confusion. After further investigation, I found an example which caters for the use case we've been discussing so far, that is the nice-url example.
The nice-url example, after mounting the
Home page as "/the/homepage/path", simply adds a BookmarkablePageLink to the Home class on another page, namely Page1. Here is the relavant code.<a wicket:id="homeLink" href="#">[go back]</a>which magically works, in the sense that when the link for the url is rendered, it will somehow contain the mounted path "/the/homepage/path". How does
public class Page1 extends WicketExamplePage {
public Page1(PageParameters parameters) {
add(new BookmarkablePageLink("homeLink", Home.class));
}
}
BookmarkablePageLink know that Home is mounted on "/the/homepage/path"?It does not know. Instead, it invokes the
urlFor(IPageMap pageMap, Class pageClass, PageParameters parameters) Component. This method will in turn ask the request cycle the page is associated with to find the correct url. The request cycle will eventually go through all mount points, asking them whether they apply to Home.class. The first mount point which applies will then be asked to encode the Home page, returning the mounted path. The exact details might vary, but from a 100 feet point of view, wicket does reverse lookup to see which mount point maps to a given page which goes to explain the aforementioned magic.
Friday, October 16, 2009
IDEA open sourced
Coming across Cedric Beust's comments on IDEA going open-source, I thought I'd chime in. Cedric makes valid points, in particular about the unfulfilled expectations after open sourcing a product. As he points out, Jetbrains is unlikely to be flooded with patches and additions from the community, floating them back to the top.
Cedric also correctly identifies that open-sourcing software is an attempt to regain mind-share. He unfortunately qualifies the effort as "last-ditch" which is where I disagree. Companies have to make strategic decisions all the time. Companies live and die by strategic decisions. Who would have guessed 10 years ago that IBM would have survived its transition to services as decided by Louis V. Gerstner? Instead of "last-ditch", in my humble opinion "just-in-time" would be a better qualification for Jetbrains' recent decision.
According to scientific surveys conducted around a glass of beer, people who use IDEA on a daily basis absolutely love it. I am not one of them. But when a company enjoys such a dedicated following, it seems premature to predict its imminent doom.
As Cedric put it, I wish the best to IDEA. I really do.
Cedric also correctly identifies that open-sourcing software is an attempt to regain mind-share. He unfortunately qualifies the effort as "last-ditch" which is where I disagree. Companies have to make strategic decisions all the time. Companies live and die by strategic decisions. Who would have guessed 10 years ago that IBM would have survived its transition to services as decided by Louis V. Gerstner? Instead of "last-ditch", in my humble opinion "just-in-time" would be a better qualification for Jetbrains' recent decision.
According to scientific surveys conducted around a glass of beer, people who use IDEA on a daily basis absolutely love it. I am not one of them. But when a company enjoys such a dedicated following, it seems premature to predict its imminent doom.
As Cedric put it, I wish the best to IDEA. I really do.
Wednesday, September 02, 2009
How the US is burning the furniture without even realizing it
The US budget deficit has reached astronomical proportions. Admittedly, the same holds true in European countries. However, European economies attract less of the world's savings than the US does.
According to various sources, in 1945 only 10% of the US government's spending was deemed incompressible, in 2009 approx. 80% of government spending is qualified as such. Given that it is politically unacceptable to raise taxes, and given that by definition incompressible spending cannot be reduced, the US government needs to find alternative ways of reducing its deficit in order to keep the ballooning budget deficit under control. If not, the whole financial system is menacing to crumble. One politically cheap way to raise money is to go after tax evasion.
In the latest revision of the Greenbook, the Obama administration proposes new measures to track down and eliminate tax evasion. To keep it short, various financial institutions referred to as "Qualified Intermediaries", even if they are based outside the US, if they serve US persons or deal in US titles (shares and derivatives), are now required to abide by strict and cumbersome rules. For a bank and its clients based outside the US, it is increasingly impractical to deal in US titles or the US banking system in general.
In addition to the Greenbook, the Patriot Act makes it unpleasant to have a banking account in the US. For example, when a person living, say in Denmark, wants to transfer funds from her US account to an account in her name at her home town in the Denmark, she has to justify the transfer with a signed letter (email won't do). From the US view point this is justified by the need to combat terrorism. However, from the view point of that Danish person living in the Denmark, the administrative overhead is cumbersome at best, and plain ridiculous at worst.
For a country which in the past managed to attract 85% of world-wide savings, the Patriot Act combined with latest steps described in the Greenbook, carefully engineered to stop tax evasion under the "no escape" principle, instead of raising more money for the government, are likely to precipitate America's continued impoverishment.
The irony of it all is that various governmental actors are all acting rationally in the best interest of the US but the end result might well be a new world order with the US playing a lesser role.
According to various sources, in 1945 only 10% of the US government's spending was deemed incompressible, in 2009 approx. 80% of government spending is qualified as such. Given that it is politically unacceptable to raise taxes, and given that by definition incompressible spending cannot be reduced, the US government needs to find alternative ways of reducing its deficit in order to keep the ballooning budget deficit under control. If not, the whole financial system is menacing to crumble. One politically cheap way to raise money is to go after tax evasion.
In the latest revision of the Greenbook, the Obama administration proposes new measures to track down and eliminate tax evasion. To keep it short, various financial institutions referred to as "Qualified Intermediaries", even if they are based outside the US, if they serve US persons or deal in US titles (shares and derivatives), are now required to abide by strict and cumbersome rules. For a bank and its clients based outside the US, it is increasingly impractical to deal in US titles or the US banking system in general.
In addition to the Greenbook, the Patriot Act makes it unpleasant to have a banking account in the US. For example, when a person living, say in Denmark, wants to transfer funds from her US account to an account in her name at her home town in the Denmark, she has to justify the transfer with a signed letter (email won't do). From the US view point this is justified by the need to combat terrorism. However, from the view point of that Danish person living in the Denmark, the administrative overhead is cumbersome at best, and plain ridiculous at worst.
For a country which in the past managed to attract 85% of world-wide savings, the Patriot Act combined with latest steps described in the Greenbook, carefully engineered to stop tax evasion under the "no escape" principle, instead of raising more money for the government, are likely to precipitate America's continued impoverishment.
The irony of it all is that various governmental actors are all acting rationally in the best interest of the US but the end result might well be a new world order with the US playing a lesser role.
Saturday, August 15, 2009
GIT migration
A few years ago, I watched Linus Torwald's presentation on git. I was so thoroughly put off by his style that I did not give git another thought for several years, that is until recently. Hearing more and more about git, I recently raised the question of migrating the logback project to git on our mailing list and the ensuing discussion had me convinced.
Here is a recount of the steps I've taken to migrate logback to git from SVN.
First, I installed git on our Linux servers. Nothing newsworthy there. Gitosis' installation on those servers was equally uneventful.
Installing git on Windows was more of an ordeal. Supposedly, on Windows git is officially supported with cygwin, which I've been using daily for over 10 years. Git just did not work me with cygwin. Migrating to cygwin 1.7-beta was not much of a help. Fortunately, the msysgit project provides a solution for Windows.
The next step was to create a git repository from our existing SVN repo which was as easy as invoking "git svn --stdlayout --no-metadata -A authors.txt clone http://svn.qos.ch/repos/logback" where authors.txt is a file mapping login names to user names. The --no-metadata flag tells GIT to drop noisy references to the SVN url and other useless details.
My next step was to allow anonymous reading of the git repository, which as easy as starting git-daemon which was part of the core git package (installed by gentoo).
Git makes it very easy to fork projects. On github, you can track work done on forks, which was one of the arguments that convinced me to switch to git hub. However, pushing to github is slower than pushing to git.qos.ch, our dedicated git server. The idea is to push to git.qos.ch on a regular basis and to githib with lower frequency (say once a week). To add a remote server, the command is "git remote add github git@github.com:ceki/logback.git". Once github is added as a remote server, pushing to it is as easy as invoking "git push github master".
Now that the essential pieces are in place, the next step was to better understand git's philosophy by reading the Pro Git book.
After much of the initial migration was done, Jukka Zitting observed that the branches and tags which existed in the SVN repository were missing. Since I had removed the original git repo which was created from SVN, I had lost the branch and tag information. I had to re-import from SVN. Since the initial import contained metadata which I no longer wanted, my newly imported repository was incompatible with the earlier git repository containing new work but missing the tagging information. I did not want to waste half-a-day's work. I saved the new work with the ' git format-patch -o ~/patches/ 088f.." command, where 088f.. stands for the SHA-1 of my earliest commit since the SVN migration. Such information is very easy to obtain from the git.qos.ch and github servers.
The next step was to create the tags from the latest git repository. Since tags can be created a posteriori, all you need is the SHA-1 of the snapshot you want to tag. The "git for-each-ref refs/remotes/tags" does give you exactly that. Here is sample output:
Which can be easily massaged into:
Once the tags are created locally, they can be pushed remotely with "git push origin --tags"
Here is a recount of the steps I've taken to migrate logback to git from SVN.
First, I installed git on our Linux servers. Nothing newsworthy there. Gitosis' installation on those servers was equally uneventful.
Installing git on Windows was more of an ordeal. Supposedly, on Windows git is officially supported with cygwin, which I've been using daily for over 10 years. Git just did not work me with cygwin. Migrating to cygwin 1.7-beta was not much of a help. Fortunately, the msysgit project provides a solution for Windows.
The next step was to create a git repository from our existing SVN repo which was as easy as invoking "git svn --stdlayout --no-metadata -A authors.txt clone http://svn.qos.ch/repos/logback" where authors.txt is a file mapping login names to user names. The --no-metadata flag tells GIT to drop noisy references to the SVN url and other useless details.
My next step was to allow anonymous reading of the git repository, which as easy as starting git-daemon which was part of the core git package (installed by gentoo).
Git makes it very easy to fork projects. On github, you can track work done on forks, which was one of the arguments that convinced me to switch to git hub. However, pushing to github is slower than pushing to git.qos.ch, our dedicated git server. The idea is to push to git.qos.ch on a regular basis and to githib with lower frequency (say once a week). To add a remote server, the command is "git remote add github git@github.com:ceki/logback.git". Once github is added as a remote server, pushing to it is as easy as invoking "git push github master".
Now that the essential pieces are in place, the next step was to better understand git's philosophy by reading the Pro Git book.
After much of the initial migration was done, Jukka Zitting observed that the branches and tags which existed in the SVN repository were missing. Since I had removed the original git repo which was created from SVN, I had lost the branch and tag information. I had to re-import from SVN. Since the initial import contained metadata which I no longer wanted, my newly imported repository was incompatible with the earlier git repository containing new work but missing the tagging information. I did not want to waste half-a-day's work. I saved the new work with the ' git format-patch -o ~/patches/ 088f.." command, where 088f.. stands for the SHA-1 of my earliest commit since the SVN migration. Such information is very easy to obtain from the git.qos.ch and github servers.
The next step was to create the tags from the latest git repository. Since tags can be created a posteriori, all you need is the SHA-1 of the snapshot you want to tag. The "git for-each-ref refs/remotes/tags" does give you exactly that. Here is sample output:
a8f066010596dc.. commit refs/remotes/tags/SLF4J_1.3.0
260b5f6f9d0f14.. commit refs/remotes/tags/SLF4J_1.3.1
d8eadb09a11dbd.. commit refs/remotes/tags/SLF4J_1.4.0
7011d1f3665e88.. commit refs/remotes/tags/SLF4J_1.4.1
5fd428e3dd03ed.. commit refs/remotes/tags/SLF4J_1.4.2
04a8c2c2cf4230.. commit refs/remotes/tags/SLF4J_1.4.3
Which can be easily massaged into:
git tag -a SLF4J_1.3.0 a8f066010596dc -m "SLF4J_1.3.0"
...
...
Once the tags are created locally, they can be pushed remotely with "git push origin --tags"
Tuesday, July 14, 2009
The quest for an AC Adapter for my Dell laptop (Part II)
The UPS delivery person left an InfoNotice(SM) at our door step on Friday the 10th July. The InfoNotice contained a tracking number which could be used at the UPS web-site to change the parameters of delivery. One of the proposed options was to return the package to the sender, which I quickly selected. By returning the erroneous package, Dell would get their merchandise back, proceed to reimburse me and the matter would be settled.
I was quite surprised to find a second InfoNotice in my mailbox on Monday the 13th. I called up UPS to learn why my cancellation request had no impact. The UPS customer service representative explained that UPS had a special deal with Dell. Only Dell could cancel a delivery; the recipient, the end-customer in this case, was not authorized to cancel a delivery. I called Dell to ask them to cancel but was told that canceling policy was an internal UPS matter. When I explained that UPS had told me to call them (Dell) because they were the only party which was able to act on the matter, the answer was invariably: "it is an internal UPS matter, call UPS".
So I called UPS again to explain the situation. The UPS representative told me that Dell was purposefully denying their capability to resolve the matter because they did not wish to cancel the order.
Since I am at the office during the day, I asked UPS if I could change the delivery address to my office, which is one of the options explicitly mentioned on the InfoNotice. The UPS representative told me that only Dell could change the final delivery address.
To be continued...
I was quite surprised to find a second InfoNotice in my mailbox on Monday the 13th. I called up UPS to learn why my cancellation request had no impact. The UPS customer service representative explained that UPS had a special deal with Dell. Only Dell could cancel a delivery; the recipient, the end-customer in this case, was not authorized to cancel a delivery. I called Dell to ask them to cancel but was told that canceling policy was an internal UPS matter. When I explained that UPS had told me to call them (Dell) because they were the only party which was able to act on the matter, the answer was invariably: "it is an internal UPS matter, call UPS".
So I called UPS again to explain the situation. The UPS representative told me that Dell was purposefully denying their capability to resolve the matter because they did not wish to cancel the order.
Since I am at the office during the day, I asked UPS if I could change the delivery address to my office, which is one of the options explicitly mentioned on the InfoNotice. The UPS representative told me that only Dell could change the final delivery address.
To be continued...
Monday, July 06, 2009
The quest for an AC Adapter for my Dell laptop (Part I)
The AC adapter for my laptop from Dell recently had its cable severely twisted so as to stop charging the laptop. The adapter is marked as belonging to the PA-9 family. I called Dell technical support and explained the problem to a technician. I gave him the serial number of my laptop which identified both my laptop and the buyer (me). He said that they could ship a replacement adapter forthwith. I asked the technician several times to confirm that item he was sending me was a PA-9. He said that it was indeed, and that I had nothing to worry about.
A few hours later, I received a written confirmation for my order. Lo and behold, the order was for a Kensington 450-11303 an adapter for the cigarette lighter of a car. I immediately wrote back to cancel the order. When Dell support wrote back, they claimed that the order had already shipped and could not be canceled. Moreover, Dell was not willing to take any further action, such as recognizing their error, before the shipment arrived. I am afraid that their strategy for dealing with this issue will be to drag it on until I give up. Anyway, time will tell.
A few hours later, I received a written confirmation for my order. Lo and behold, the order was for a Kensington 450-11303 an adapter for the cigarette lighter of a car. I immediately wrote back to cancel the order. When Dell support wrote back, they claimed that the order had already shipped and could not be canceled. Moreover, Dell was not willing to take any further action, such as recognizing their error, before the shipment arrived. I am afraid that their strategy for dealing with this issue will be to drag it on until I give up. Anyway, time will tell.
Subscribe to:
Posts (Atom)