Aehso’s Output

RailsConf Europe Day 1

[Update: Some of the presentation files are now available on the RailsConf Europe website]
Some quick notes from some of the excellent conference sessions that I attended yesterday at RailsConf Europe.  I wasn’t here for the Tutorials on Monday though now I wish I had been - the quality of the presentations (at least the ones I’m picking) is excellent.  Aside, Berlin is a cool spot, I must come back here sometime again to have a decent look around…

Deployment and Continuous Integration from the Trenches, (Fernand Galiana - LiquidRail)

• All about latest developments for Capistrano 2.0 - it seems to have matured considerably in recent months.
• Use multistage_ext
• Use shared project capfiles to keep things DRY
• Use remote repository cache to speed up deployments
• Read Jamis Buck’s blog and the Capistrano Google Group

Really Scaling Rails, (Britt Selvitelle - Twitter)
I’ve seen presentations on Twitter scalability, and even since then they have had a few more high profile outages (at least high profile amongst Twitter users). A couple of interesting takes from this talk:

• Twitter uses Apache, mod_proxy_blancer and mongrel servers
• All user traffic is handled by a single MySQL server. That server does have a slave that can be promoted to master (for redundancy). They have a couple of other MySQL databases for use internally for reporting etc.
• They set mongrel’s num_procs to 1. This means that each mongrel server instance will not queue requests from mod_proxy_balancer - they will only accept one at a time. The side effect is if the concurrent request count > mongrel server count then users start getting error pages. Strangely, they’d rather users got errors than risking loosing queued requests whenever they have to restart a mongrel instance (mongrel apparently waits only 60s before sending itself a TERM signal when shutting down).

Rails Full Text Search with Ferret

A little different to what I was expecting, Ferret provides an indexing service for arbitrary strings (documents), similar to Apache Lucene in many respects.  Worth a look, if you need to support full text search within your rails application.

Tabnav: Do We Really Need a Plugin for Tabbed Navigation? (Paola Dona - Seesaw)

The title of this one was a a bit modest as Paola presented a slew of new Rails Widgets that SeeSaw have developed, all of which seem to be very flexible. Widgets for include tabbed nav (or course), site nav, tables (blocks), nubbins, show/hide blocks, help popups were all demoed. Their widgets integrate very nicely into the rails views/templates - all in all, it looks at least good enough for use in creating rails app prototypes and it appears they might be flexible enough to be embedded into a production application…
The Rest of REST (Roy T. Fielding) - slides

Good historical view of where Roy came from and how the principles of REST have always been such an important underpinning of the IETF’s thinking behind standardization of key web specifications like HTTP, URI and HTML. He provided a good overview of how the web architectural style is defined as a set of constraints:

• client server
• stateless
• caching
• uniform interface
• layered systems
• code-on-demand

There was some interesting discussion on what is missing from Rails though Roy’s first two suggestions drew comments that he might have missed features in Rails that do what he wanted.  His last suggestion, for Rails to guide the developer into using hypertext as the engine for of application state (man they really have to find a shorter name for that!) was a fair comment - seems like an incredibly difficult problem to solve in a generic framework like Rails but as he said, it’d be a first…
Last talk was Craig McClanahan finishing off the day with a short Rails and the Next Generation Web pitch.  Now I hadn’t realised Craig had switched from Java to Ruby development and he seems to be loving it.  Craigs name has all over the Apache code I’ve worked with for years now - he was one of the original Tomcat Catalina & Struts developers and he also co-authored the Servlet and JSF specifications.  He had an interesting anecdote about how the Struts developers all suddenly found themselves working on non-struts based projects…

Model/View/Template or Model/View/Controller?

Bill de hÓra has a nice piece on Struts 1 that touches on something about MVC that I’ve never quite liked in the web application context:

In pattern language terms, the MVC pattern has been reapplied to the Web domain without consideration for the forces induced by the Web. MVC frameworks tend to be inwardly focused contraptions of server sided redirects. Alternatives: On the web, a suitable pattern is View, Model, Template. A request to a URL is dispatched to a View. This View calls into the Model, performs manipulations and prepares data for output. The data is passed to a Template that is rendered an emitted as a response. ideally in web frameworks, the controller is hidden from view.

I like the View, Model, Template pattern suggestion, the separation of View (which data can be displayed) and Template (how it gets displayed) is an important one that I think too often gets lost in web application development.

If only the framework could be the entire controller all on its own though! Even the Ruby On Rails (the poster boy web application framework of the moment) cannot do everything unaided.  Writing controllers is not too bad if you are exposing RESTful resources - much custom controller logic is then reusable though things like authentication still intrude.  RPC oriented web appications are the ones that can really end up suffering from horrendously complex controller definitions.

The other main difference between Java/Struts1 and something like RoR is the considerable benefit gained by being able to define the controller completely within in the host language rather then having to resort to a code+XML definition.  I don’t think this should be underestimated. Personally I have always found the biggest impediment to grokking, testing and maintaining Java/Struts1 application code was the mixed Java/XML implementation, and the IDE support it almost demands when used in anger.

BTW, big contratulations to Bill on getting the APP spec (nearly) finished!

Glassfish V3 HK2 Modules

Glassfish V3 introduces a new kernel, HK2 (Hundred Kb Kernel) , that has yet another modules model. An interesting statement on the H2K home page is tantalizing:

Modern Java software usually rely on a module subsystem to offer a better level of isolation between parts of the application. Many technologies have been around for years, HK2 proposes a model which is aimed to be friendly to existing technologies such as OSGi yet will provide a path to the implementation of modules in Java SE 7.

Scant details on the OSGi integration/adaptation layer so far, but it’ll be interesting to see how this one pans out…

The dreaded “java.lang.OutOfMemoryError: PermGen space” exception.

Frank Kieviet has two excellent posts that explain the dreaded “java.lang.OutOfMemoryError: PermGen space” error and more importantly how to use some new Java 6 tools to diagnose the source of the problem. The first post in particular highlights a very subtle issue with classloader/reference handling that anyone developing code for multiple-classloader environments needs to be aware of.

Alessandro Ribeiro also has an interesting three post piece where he shows how to use the JDK 1.5 ‘jps’ and ‘jstat’ tools to help diagnose the same problem (hint: “jstat -gcpermcapacity ” will give you a nice summary of the status of the VM with the target PID).

Users have been running into this when running Eclipse-based products on Sun’s JDK 1.5 for some time and Eclipse Bug 92250 has been quietly collating lots of feedback and suggestions from the Eclipse community along the way. A lot of people have been lurking/waiting/hoping that someone would identify the source(s) of the issue and produce a fix so it is good to see increased activity there. However, the recent debate about whether this is a Sun or Eclipse (where I found links to the above blog posts - thanks Karsten!) issue might be a little discouraging but it appears there is acknowledgement that some responsibility lies within the Eclipse community to help diagnose the problem.

I’d imagine many adapters would be willing to lend a hand in diagnosing this since it is a potential stability issue for all Eclipse based products when run on what is the de-facto Java VM. However, I suspect any such analysis effort would need to be done in coordination with some platform/WTP folks since they would need a good insight into the expected behaviour of all of the active bundles to be able to determine which are at fault (or if indeed the fault lies with the VM)

What intrigues me about this issue is that Eclipse/Equinox doesn’t normally destroy classloaders so classloaders not getting garbage collected can’t be the source of the problem. But with that assertion, how can any application be hitting a 512Mb permGen limit? It does smell of a VM bug but then why doesn’t it happen with other Java based applications? Clearly some feature common to many Eclipse applications is rubbing the Sun VM up the wrong way.

I guess the only way to find out is to do a very deep analysis but a pre-requisite for that is to find a simple use case that can reliably and repeatedly reproduce the problem. We don’t seem to have one with our product but perhaps someone else out there does.

What happened to JAX-WS 2.1 & JAX-B 2.1?

Update: In my dismay I was getting the JCP & W3C processes all mixed up - duh. It seems it was the W3C WSDL Binding specification that fell at the last hurdle, presumably causing the JAXB 2.1 and JAX-WS 2.1 specifications to be withdrawn.

Update2: Vivek Pandey’s Blog post seems to confirm the above. I’m all for blogging to get the word out to the masses but the JCP Expert Group(s) should update their official JSR pages too. Regardless, the fact that JAX-WS 2.1 JAXB 2.1 was finalized on a Candidate Recommendation W3C specification reflects poorly on the quality of specification that the JCP is producing.

Does anyone know what happened to JAX-B 2.1? JAX-WS 2.1 (JSR-224) was released a few weeks ago, but the JCP have since pulled the specification, reference implementation and TCK. The reason given is that the W3C WSDL Binding specifiication, referenced by the JAX-WS 2.1 JAXB 2.1 specification announced back in December, did not make it past the Candidate Recommendation stage and apparently needs to be replaced with an alternative specification.

Does anyone have any pointers on why this happened (and what the old JAX-WS 2.1 candidate might be replaced by or when)?

Update 3: Looks like WS-Addressing WSDL Binding is being replaced by WS-Addressing Metadata.

Eclipse in a browser - Eclipse Rich Ajax Platform (RAP)

I’ll not be at EclipseCon this year (it conflicts with polishing our next major product release this week) but if I was there I’d be going to any sessions related to the new Rich Ajax Platform (RAP) incubator project. Think the Eclipse UI running in your browser, with the Eclipse application, developed as plugins, running remotely on a web server.

The Yoxos Eclipse On Demand online Eclipse distribution builder is a good example of the types of applications this framework could support. In fact, given that Yoxos is run by Innopract, the folks who are driving the RAP project, it is probably the best example out there. It’d be interesting to find out how far along they are with their RWT API implementation - attempting to build a remote client framework based on a local client framework like SWT is perhaps a dangerous thing to attempt to do but we’ll see how they fare.

(Not to be confused with the WTP 2.0 AJAX Tooklit Framework (ATF) incubator project or the Google Web Toolkit (GWT) which are designed to also support development of AJAX applications, albeit using other frameworks and user interfaces…)

JAX-WS and BPEL are maturing.

Some interesting recent developments in the Web Service space that I’ve been meaning to point to:

• JAX-WS 2.1 has been released.  Enter the tubes…
• BPEL 2.0 is now almost an official OASIS standard.  Do we now have to start referring to it as WS-BPEL instead of plain old BPEL/BPEL4WS?

Only slightly related, some interesting Java WS stacks benchmarks:

• Axis2 vs XFire, published by WSO2.  (with interesting follow-up from Hani and retort from Sanjiva Weerawarana).
• JAX-WS 2.1+JAXB vs Axis2.1.1+XMLBeans, published by Sun.

JSR-271 Early Draft reviews

The java modularity train-wreck-to-be is still ploughing full steam ahead.  The recent publication of the JSR-277 Early Draft has resulted in a some very sharp criticism.  Lots of posts aggregated in this great post but Patrick Beno’s review is one of the most comprehensive I’ve read. 

InfoQ have just gone and stoked the flames further with post where their questions (solicited from the community) met with laughable responses from the JSR-271/JSR-294 spec leads.  Honestly, the companies and individuals represented on the ECs for this JSR should be asking themselves some hard questions about whether they want to continue to associate themselves with this mess.

Sun GPL Java.

Sun’s GPL’ing of Java SE, Java ME and Glassfish is definitely interesting - Tim Bray has some interesting commentary from the inside, no doubt slightly sanitized! Kudos to Sun for going GPL, thereby enabling it to be shipped with the GNU/Linux distributions. I am sure there are those in the OSS community who will remain suspicious.

Plenty of other questions yet to be answered, the governance model of the new OSS projects will be critical - it’ll be interesting to see that is set up (Eclipse.org style perhaps?). Interesting to note that Sun are retaining the TCK code (to retain control of Java brand compliance) and they are not changing the JCP. The latter decision is very interesting as if the specification process is still tightly controlled by Sun then that leaves limited latitude for the implementation project to go in other directions.

Once the dust settles, Java ISVs around the globe should be ready to re-validate their software platforms and distributions. Questions now exist about the viability of some GPL-licensed J2SE, ME and EE projects and their future viability. It is bound to impact Apache Harmony , a project largely driven by IBM (who predictably have reacted by suggesting that Sun should contribute their Java technologies to Apache.org - haha). And no doubt this will make GlassFish more competitive with JBoss. I’m sure this also impacts the ME space - interesting to note that the GPL nature of the license means any derivatives for other embedded platforms also have to be GPL’ed.

Interesting times…

ApacheCon Europe ‘06 summary

So ApacheCon Europe ‘06 has been on in the Burlington for the past few days )anyone wandering around Donnybrook/Leeson Street in the mornings/evenings over may have noticed a “geek increase”. It was a bit smaller than I expected but the quality of the content was superb if one bears in mind that these guys are doing this stuff part time (well, some of them!). The sessions I attended were all SOA/OSGi centric and all were worth attending, from an educational point of view. A very partial summary:

• The OSGi/JSR#291/JSR#277 issue is still very much alive although some efforts are being made to try achieve some degree of compatibility by using a common subset of Manifest headers. However differences apparently already exist in the drafts (such as the version range specifier format used) It is pretty sad to see such open (or should I say JCP-closed) disregard for interoperability between two overlapping JSR. Both are due for delivery in Dolphin (Java 7), perhaps by then the whole Java platform will have been rendered unusable by the plethora of other overlapping JSRs and we (the users) will all have moved on to coding in Ruby.
  Actually, by then, it may not matter - OSGi seems to be gaining considerable traction. Tuesday afternoon had a whole afternoon of talks and discussion from Richard Hall (Felix), Peter Kriens (OSGi Alliance/aQute), Marcel Offermans (Luminis) and a round table discussion involving folks from Apache Maven, Felix, Harmony, Directory Server projects. It seems some efforts are being made to try get Apache Jakarta projects to OSGi-fy their Manifests (via capabilities built into Maven)
• Woden looks good. It’ll be the WSDL2.0 processor that all Java folks end up using (if you need to parse WSDL2.0 that is!). It replaces WSDL4J which is being put out to pasture (it must be, I logged a bug against it over a year ago and still no feedback!)
• Axis 2 apparently has WSDL2.0 HTTP binding support with some restrictions (messages must be “IRI style” compatible). I’m not sure if one even needs to declare a HTTP binding in the WSDL, it seems to be automatically available for all services that have SOAP endpoints.
• Apache Synapse looks nice, in a cute kind of way. I’m not sure I’d call it an ESB since it is really just a very thin routing framework that runs on top of Axis 2 - it doesn’t have any management interface per say and the overview might have been a bit forward looking with regard to the capabilities of the current implementation Synapse looks more like an endpoint intermediary that you one might use to implement specific tasks (XPath/RegExp based routing, binding conversion, logging, endpoint authentication & authorization, transformation (XSLT, E4X, POJO based). I’m not sure you’d host your service implementations on it.
• Apache Tuscany work is ongoing and I bet it will be for some time yet! I’m not certain of their claim that it will simplify the development of business solutions, if only because it is based on SCA which is turning into an absolute beast of a set of specifications - think “one spec to rule them all” type big, and with big specifications comes complexity, not simplification. From an implementation point of view, a huge problem is that the SCA specifications do not have either a reference implementation nor a compatibility test suite and according to Simon Nash and Jeremy Boynes (both of IBM) there are no current plans to develop either. Also it seems curious that the SCA specifications are not being developed under the auspices of an open body like OASIS, W3C or the OMG - why not? Some would say the overall approach seems incredibly vulnerable to repeat the mistakes of CORBA. It will also be interesting to see if any convincing response is given to Ron Ten-Hove’s recent critique of SCA.