Yet another choice

Recent adoption of the SLF4J API by Tapestry and Howard's blog on the subject has triggered a frenzy of comments, most of which were very favorable with the exception of Dion Almaer. Dion ridicules the unholy habit we J2EE developers have of trying to abstract every little API we might come into contact with.

I am inclined to agree with Dion but for different reasons. Writing a good abstraction layer for two or more distinct systems takes serious effort. I'd go as far as declaring that the task is impossible unless the systems in question are very similar or owners of these systems unconditionally submit to the authority of the abstraction layer.

In the case of log4j and java.util.logging (JUL), Jakarta commons-logging (JCL) was only able to partially abstract the underlying APIs because their core APIs are similar both conceptually and structurally. However, JCL was not able to abstract parts below the core API. For example, the JCL does not offer any help with respect to configuration of the underlying logging system. SLF4J fares only a little better, in that it offers abstractions for both MDC and Marker, in addition to the core logging API.

JDBC can be cited counter example of a successful abstraction layer. However, it is successful insofar as the RDMS providers submit to the authority of JDBC specification. They all go out of their way to implement a driver compatible with the latest version of the JDBC specification. Moreover, RDMS applications already share a similar structure by way of SQL.

When the systems differ substantially, it is nearly impossible to bridge the gap. Is there an abstraction layer bridging relational and OO databases? I think not. The relational/OO impedance mismatch gave birth to major development efforts. Take Hibernate for instance. Would you dream of writing Hibernate as a weekend project?

So why did JCL, with all its warts, catch on like wildfire? Because JCL provides a convenient answer to the log4j vs. JUL dilemma faced by authors of most Java libraries. The dilemma does not exist in other languages because there usually is one predominant logging system for the language. In Java we have log4j getting most of the mindshare, with JUL looming in the background, not much used but not ignorable either -- hence the dilemma.

Anyway, Dion has a point. We, in the J2EE community, do indeed waste too much time dabbling in secondary matters such as logging, but we only do so because we have the luxury of choice. We can chose between log4j, logback or JUL as our logging system. We can choose between Ant, Ivy or Maven for our builds. We can choose between Eclipse, IDEA and Netbeans for our IDE. We can choose between JSF, Tapestry, Spring, Struts or Wicket as our web-application framework.

Making choices takes time and effort but it also exerts a powerful attraction on our psyche. When presented with the choice, programmers (to the extent that we programmers can be assimilated to humans) will prefer the situation where we can choose between multiple options than the situation when we are presented with only one option.

Java presents us with more choices than any other language, probably because it is also the most successful language in history. Of course, you already know that successful does not necessarily mean best.

Anyway, I am quite happy see SLF4J being adopted so massively.

GIT vs Subversion

Linus Torvalds recently (2007-05-05) gave a presentation about GIT at Google. The video of the presentation is available on youtube.

In this particular presentation, I found Linus to be opinionated and rather unconvincing. He is extremely critical of CVS and Subversion. While GIT may be well-adapted to Linux's development model, I believe Subversion get the job done in other environments.

Martin Tomes, in his comments about GIT, nails the point. GIT and Subversion aim at different development models. While not perfect, the classical (centralized) model works well in both large and small projects, open-source or not.

The GIT project publishes a detailed albeit biased comparison between GIT and Subversion. The comparison makes a convincing case on why GIT offers better support for merges. The same page also mentions that the user interface for Subversion is better.

Selling YAGNI

I am quite fond of the YAGNI principle because it helps me concentrate on the essentials of the application currently under development. Another explanation is that I am getting lazier with age.

YAGNI tends to sell well with developers. It prunes needless work. However, with customers who ask for features, the YAGNI principle does not sit quite as well. People in general do not appreciate their decisions to be questioned and YAGNI can be resumed to one question. "Do you really need this feature?" The answer is often yes, forcing the skeptic in me to repeat the question in perhaps a modified form. Most people, customers included, do not like to be challenged, especially if done with some insistence.

Pruning requirements to mere essentials takes both work and courage. In the eyes of the customer, the alternative, i.e. asking for potentially useless features, may often look both easier and less risky.

I try to use the arguments as advocated in the c2 wiki, the feature that is implemented by anticipation now may be radically different than the feature actually needed in the future.

So how do you apply the YAGNI principle in a real-world environment? What are the arguments that may sway your customers or fellow developers?

Evolving a popular API

Authoring an API which later becomes popular can be both a blessing and a curse. If your design was imperfect, which it is bound to be, you will be frequently flamed for flaws. Except for the most trivial systems, it is outright impossible to get an API right the first time. You will several iterations to perfect your design.

Take Tapestry for example. It has evolved over seven years and five iterations to become what it is today. Unfortunately, some of these iterations were not backward compatible, thus purportedly negatively impacting Tapestry's adoption rate.

Offering a painless migration path to users may be necessary element to keep your existing user base, but as any developer who has attempted to preserve 100% backward-compatibility will tell you, such an ambitiuos goal will quickly begin to consume eons of your time.

Unless you are Microsoft or some other entity with serious resources, you will need to make a choice between 100% compatibility and innovation. In my experience, you can't both improve your design and keep 100% (absolute) compatibility.

However, if you aim a little lower than 100%, you can keep evolving your API without severely impacting your existing users. Most APIs have parts intended for internal use and other more public parts intended for use by the wider public. Changes to the internal parts may effect a handful of users, say one out of every thousand users. In contrast, changes to the public API will affect every user.

If such a distinction makes sense for your API, confine your incompatible changes to the internal part of your API. As mentioned earlier, these changes may affect a small proportion of your users, which may still number in the hundreds. Nevertheless, causing discomfort to a tiny minority of your users is still much better than a dead, i.e. non-evolving, API.

Dell delivers (not!)

My company has been a Dell customer for many years, having purchased four computers in the last 12 months alone. A few weeks ago we decided to purchase a new laptop, more precisely Latitude D620. This baby comes with an Intel Core 2 Duo clocked at 2.0Ghz, a 14" inch screen with a resolution of 1440x900 pixels. Most importantly, it weighs 2.0Kg (4.4lbs).

We signed the order on the 30th of April 2007 and paid for it on the 3rd of May. Tracking the order on Dell's web-site, we noticed that the order was not being processed. I contacted the sales person to inquire about the order. She said that as fat as she could tell no payment was received and that she needed proof of payment to look into the matter. After sending her proof of payment, it took another day for the accounting team to match our payment with our order. Nevertheless, with the payment glitch fixed, the laptop went into preproduction on the 9th, was finished the next day and shipped by UPS on the 11th with expected delivery on Wednesday the 16th of May.

Lo and behold, we received it on the announced date, at around 11 AM. I was quite excited to receive this new laptop as a replacement for my older Inspiron 5100 (also from Dell). After 4 years of good and loyal service, while my old companion still works nicely, it weighs a hefty 3.5kg (7.7lbs). Since I have to schlep it on foot for about an hour each work day, 1.5kg (3.3lbs) less weight on my back is something I was looking forward to.

Opening the package, all the components were there. Unfortunately, instead of weighing 2.0kg (4.4lbs) my Latitude D620 weighs 2.5kg (5.5lbs), that is a 25% difference compared to my order and Dell's own specifications. Contacting the sales person, she proposed to sell me 4 cell battery, purportedly lighter than the 6 cell battery I currently had. Unconvinced, I asked to speak to her manager and somehow got disconnected. Sigh.

The second time I called I was put in contact with a customer service representative, who, recognizing the problem, promised to replace my laptop with a model of my choice. Needless to say, I was quite impressed by Dell's generous offer. To good to be true, she called an hour later reneging on her previous offer, under a completely bogus pretext. Let me cut a long story short by saying that there is a limit to the amount bull this particular customer (yours truly) was willing to put up with.

How can Dell hope to retain customers when what they deliver only approximates what they advertise ? One of the customer support people at Dell went as far as acknowledging that Dell was minimizing their laptop's weight to increase sales and that other vendors were also playing the same dubious game. One thing is for sure, we won't be buying another Dell product anytime soon.

Reading XFire code

The various SOAP and WS-* related specifications have the reputation of being tricky and difficult to understand. The latest project I am involved in requires a relatively deep understanding of WS-*. One way to gain understanding of a specification is by closely studying an implementation of it. Spurred by my previous pleasant experience with it, XFire happens to be that implementation of choice.

Anyway, checking out the project from SVN and building from the sources was fairly easy. A number of tests failed, but all in all, it was a breeze to get the various XFire projects nicely tucked under Eclipse.

From what I can tell, the code is a pleasure to read and it feels like it is the result of fairly good design.

Founders at work

After reading the first 3 chapters of "Founders at work" by Jessica Livingston, I can't help but recommend this book. Compared to many other books where the fluff in the narrative ends up diluting the content, the direct language of the various founders is both refreshing and inspirational. Each story if filled with unsophisticated yet brilliant ideas, each resembling a small gem.

I can't wait to read the remaining chapters.

SLF4J and logback gaining traction

It does not mean much me saying so, but the SLF4J and logback projects are gaining traction. The project mailing lists are showing real signs of life and community interest, whereas the download statistics are showing significant upward trend.

We are not at the same levels of popularity as commons-logging or log4j. Nevertheless, it is very encouraging to see users responding favorably to our work. It feels like the early days of log4j, and that's pretty damn exciting.

Advantage of open source

I recently had to use two very comparable products, one open-source and the other closed-source. While the closed-source product had more verbose documentation, but I actually managed to get the open-source product running and not the closed-source product.

More importantly, the API of the closed-source product, while very similar and accomplished the *identical* task, felt awkard. I guess that bouncing ideas off users and listening to what they have to say makes a real difference at the end.


Although clearly at a commercial disadvantage, an open-source project has a structural advantage at creating a better product. Of course for really large products where the combined efforts of dozens of programmers are needed for prolonged periods, closed-source remains a valid alternative.

Release procedures

While lurking on the Maven dev mailing I came across a proposal for a release procedure. Given that the proposal is 3 pages long, in the ensuing discussion, the developers most heavily involved in the project, what one might call as the "doers", seemed to reserve a lukewarm welcome to the proposal.

There is a growing need to increase productivity of development teams, an industrialization of sorts. However, this need has to be balanced with the imperatives of creativity. Procedures affecting the day to day lives of developers need to be pragmatic and low cost. The time at the disposal of a productive developer is a scarce and expensive resource. As such, I am surprised to see Apache, an open-source icon, indulge in heavy-handed procedures. Again, it's only a proposal, and hopefully it won't be accepted in its current form.

"Release early, release often" has been one of the core mantras of open-source development. The ability to release frequently brings a level of reactivity highly appreciated by OS users. A multitude of procedures which inexorably hamper reactivity, need to be weighed against the purported benefits.

Of course, not every procedure is bad. Development teams need to be coordinated, with each developer assigned clearly defined tasks. Scrum attempts to deal with this problem. Scrum from the trenches gives an easy to read description of the approach advocated by Scrum aficionados.

Migrate from log4j to logback in seconds

It may sound insignificant but we migrated our most important
applications in production from log4j 1.3 over to logback version 0.7. We have done this without changing a single line of code in our applications but by merely replacing the file log4j.jar with log4j-bridge.jar in the relevant WEB-INF/lib directories.

Log4j-bridge intercepts calls to log4j's Logger class and transparently redirects them to logback.

I am thrilled to see that for our real-world applications previously using log4j, the migration process to logback was quick and painless.

Continuum running and configured in 20 minutes

Having been nagged by gump for ages, I've been reluctant to use a continuous integration system, at least until a few days ago. Notwithstanding my conservative attitude, colleagues have patiently and convincingly explained that having an automated system building and testing my projects, was a good thing. Taking their word, I've installed Continuum in about 5 minutes and had it configured for SLF4J and logback projects in about 15, of which most were spent entering the correct "scm" incantations in the relevant Maven2 project (pom.xml) files.

I am still not completely sold to the idea of continuous integration (CI). As I understand it, in practice, continuum will check out the latest sources from source repository, build and run the test on the CI machine, and notify our team if anything goes wrong. Already at this early stage, Continuum feels like a new member of our team. The question is whether this new member is worth the maintenance. However, from the little experience gained in the last few days, Continuum seems to do what it is supposed to do without getting in the way. A new build is done only if the contents of the source repository change, and notifications are sent only when the latest build results differ from the previous one.

In short, once you've sold your soul to M2, continuous integration via Continuum is a piece of cake.

Solution to the Maven2 version number problem

In the past few days, I've ranted profusely about the difficulty of changing version numbers of modules in Maven2. As things stand currently, when a module references its parent, it must explicitly state the parent's version in hard-coded form. Once that import is done, the natural thing to do is to define the current module's version by the version of the parent. The down side is that, when the parent version changes, this must be reflected on all child modules. If your project makes simultaneous releases of all its modules in one sweep, as many projects seem to do, then you must manually change the version number of each module by changing the version number of the parent reference in each module. This is a time consuming and error prone process at best.

I recently experimented with a solution to the above problem. It's now part of the SLF4J project (which has 10 or so modules).

The idea is to declare the version number for the whole project as a property, namely "aversion" (pun intended), in the parent pom. The parent pom's own version number can be anything as long as it ends with "SNAPSHOT".

Here is an excerpt from SLF4J parent pom:

<project>
  ...
  <groupId>org.slf4j</groupId>
  <artifactId>slf4j-parent</artifactId>
  <version>1.0-SNAPSHOT</version>
  <packaging>pom</packaging>
  <name>SLF4J</name>
  <properties>
    <aversion>1.1.0-RC0</aversion>
  </properties>
  ....
</project>

Child modules' version is specified via the ${aversion} property. Children's reference to their parent's version is hard coded. However, since the parent pom's version is a SNAPSHOT, child modules will see the changes in the parent pom. In particular, if the parent pom changes the value of ${aversion}, the children will see the change.

Here is the pom.xml file for the slf4j-api module.

<project>
  <parent>
    <groupId>org.slf4j</groupId>
    <artifactId>slf4j-parent</artifactId>
    <version>1.0-SNAPSHOT</version>
  </parent>
 
  <modelVersion>4.0.0</modelVersion>
  <groupId>org.slf4j</groupId>
  <artifactId>slf4j-api</artifactId>
  <version>${aversion}</version>
  <packaging>jar</packaging>
  <name>SLF4J API Module</name>
  ...
</project>

Unless I've missed something, this hack seems to work just fine. I would be interested to know whether there is a downside it.

Version numbers in Maven

I am probably the 50th person ranting about lack of inheritance of version numbers in Maven. Those who use Maven 2.0.4 will know what I am talking about. Maven developers plan to bring a solution with the next release, i.e. 2.1. Whatever the solution, it can't be worse than using the release plug-in, the officially recommended approach at present time. The release-plugin approach is both total crap and an insult to intelligence.

Nevertheless, I am fairly happy with Maven 2. Our 3 month investment is finally starting to pay off. If Maven were a car, it would safely take you from place to place as long as you did not switch on the radio. You see, the radio feature in Maven is not meant to be actually used. It is there for show only. As soon as you attempt to tune in to some music, the exhaust will sound off a loud bang and your car will need to be towed to the nearest garage for maintenance.

If you know of a good solution to the version number problem, please share your wisdom with us, mere mortals.

logback: a worthy successor to log4j?

As you may have already heard, I have been working on a new project
called logback, intended as a worthy successor of log4j.

On the 5th of December, I'll will be presenting (in French) the top 10
reasons for migrating your projects to logback. Issues such as
migration strategy, new APIs, SLF4J and Joran will be
discussed. Emphasis will be given to practical aspects and a live demo
rather than relatively theoretical considerations.This free-entry
event is organized by Hortis.

For those who may not be able to attend my presentation, here is abrief summary:

• Logback is an improved version of log4j
• Given that logback is built on top of SLF4J, you can switch to a another logging system at will.
• The new Joran configration API sits at the core of logback.
• The Status API for accrued resilliance. Logback's status API enables internal error reporting in a simple yet powerful way without adding complexity.
• Documentation: already good and getting better by the day.
• Filtering API. If you can imagine it, logback can filter it.
• Marker objets to color log statements for highly-specialized processing.
• Access module: easy integration with access logs generated by Tomcat or Jetty
• JMX: You can configure logback at runtime using Mbeans.
• TDD: logback has been developped test first. Moreover, logback is available for use, today.

The full presentation is also available. Anyway, I hope you will be able to attend.

Repated configuration with Joran

Developers have frequently express the need to output log files based on arbitrary runtime criteria such as by client, by task, etc.

Given all the flexibility offered by logback, writing such an appender should be easy. Let us call this new appender, MultiAppender. In principle, all MultiAppender needs to do is to create a new file as necessary according to the evaluation of incoming logging events. A configuration snippet might look like:

 <appender class="ch.qos.logback.core.MultiAppender">
  <fileNameCalculator class="ch.qos.logback.core.Calculator">
    <fileNamePattern>/some/path/%exp{userid}.log</fileNamePatttern</> 
    <expression name="userid">mdc.get("userid")</expression>
   </fileNameCalculator>
 </appender>

Thanks to Joran, logback's powerful configuration API, we can deal with unknown configuration elements such as fileNameCalculator, expression and so forth. It's a slam dunk for logback, or is it?

Although Joran can deal with arbitrary configuration instructions, it can do so only once. Assume we changed the requirements, so that MultiAppender acted like a multiplexer of appenders. Thus, instead of writing to different files, it delegated to a fully-fledged appender, according to various criteria, then MultiFileAppender would need to configure a complete appender repeatedly.

We are in the process of refactoring Joran so that it can be invoked repeatedly on parts of a configuration file. To my knowledge Joran is the only configuration system offering this capability (but I might be wrong.)

In a completely unrelated project, the same need of repeatedly configuring components came up. In this other project, we need to configure a tester, an object performing one or more tests. We create a tester, configure it, invoke its test methods, collect the results, and when done, throw the tester away to start all over again a few minutes later. We leveraged the unique capabilities of Joran to provide this particular lifecycle. Joran, part of logback-core, is a generic configuration system that you can use in your own projects to great effect.

Do ping me if you need further info,

Naked without eclipse

I recently had to make a few changes to a java project. The computer at my disposal, Gentoo Linux AMD64 machine, did not have eclipse installed. So I turned to my reliable editor/IDE, i.e. Emacs. Considering that Emacs was my IDE for several years in the past, I still felt astonishingly naked without Eclipse. Only under circumstances where a tool is withdrawn that one realizes how much comfort it brings and that you can't live without it.

Migrating to Maven 2

In the last few weeks I had to dabble in Maven, either migrating existing projects or creating new ones. The experience has been conclusive. As long as I adhere to Maven's philosophy, I am able to get things accomplished.

For instance, after the experience gained in mavenizing logback, migrating SLF4J to Maven2 from Ant has taken only few hours. The resulting project structure is imho a little easier to understand and to maintain.

As many users have observed in the past, Maven has a number of rough edges. For instance, the archetype plug-in is not well documented and can behave strangely at times.

One peeve I have is with the site plugin which has a nasty habit of stripping attributes from XML elements in xdoc files. I don't know why it decides that style classes that you painstakingly added to your document is unworthy of its consideration.

I find dependency scopes, a core feature of Maven, quite confusing. For instance, the article "Introduction to the Dependency Mechanism" mentions that the provided scope is only available on the compilation class path, while I know for fact that it is also available on the test class path. Moreover, the same article gingerly refers to several class paths without defining their meaning. Interestingly enough, the table defining the transitivity of indirectly referred projects is not properly labelled because, as mentioned previously, the xdoc plug-in has a dubious habit of stripping element attributes or even whole elements it does not like.

All the criticism aside, I am beginning to get the hang of Maven, albeit screaming and shouting. If someone tells you that Maven is a breeze, don't believe them. Build management is not an easy problem and does not suddenly become easy with Maven.

Logback version 0.1 just out the door

Sebastien Pennec recently announced the release of logback version 0.1. As the version number suggests, the logback framework still needs a lot of work. Nevertheless, we had to start somewhere...

For the next release, Sebastien and I will be concentrating on documentation and other essential features. The goal is to reach a maturity level such that developers can actually start using logback in their code.

Those skittish about committing to an unknown product can mitigate the risks by using SLF4J API, a fairly robust logging abstraction, directly supported by logback.

Maven -- whether you like it or not

As a relatively experienced Ant user, I usually try to avoid Maven, a build system intended to automate mundane and time consuming tasks such as project packaging and site generation.

Apparently, site generation is one of the most appreciated features of Maven. Given that the default site generated by Maven looks nice but otherwise is rather useless, you need to spend considerable amount of time to tame the results.

In the foreword of the book "Maven, A developer's notebook", Jason van Zyl states that "Maven is an incredibly boring technology". He goes on to say that if you use Maven, your development infrastructure will be so coherent, predictable and reproducible that you won't even think about it anymore.

Although his promise is alluring, I don't think it is fulfilled. For one, Maven is highly intrusive. It will heavily impact the structure of your project. For example, converting existing projects, even small ones, to Maven can be a painful experience. You need to adopt the Maven worldview to benefit from it. As far as Maven is concerned, it's ma-ven or the highway.

Don't get me wrong, Maven tackles the task of building projects, a very hard task to say the least. Your development infrastructure may indeed benefit from Maven. However, don't expect results without a heavy investment. Maven will cost you dearly, but so do Ant scripts. As discussed on the jakarta commons mailing lists, the question is which approach costs less. On BSF, Alan Gutierrez claims that Maven is a drag on productivity. Howard Lewis Ship explains in his blog the reasons why he has moved away from Maven. He also states (see comments section in his blog) that Maven 2 is a big imrovement over Maven 1.


Henri Yandel, who can be trusted to know a thing or two on multi-project builds, replies that Maven equals standardisation equals easy-to-understand. I think that that Henri summarizes the issue pretty well when he writes:

Slightly off-topic, if applying Maven to an Ant project, I would recommend trying to simplify your build system/code structure, don't hack Maven to fit the system you had in place for the Ant project. If you can't be standard-ish, stay in Ant - it's the right place for lots of customery.

As an alternative, you could mitigate the cost of dependency management (and only dependency management) with little known tool called Ivy which integrates well with Ant.