Dion has continued to do some very deep thinking and about *OAs and this post reflects this effort – it should be immensely useful for anyone looking at how to evolve existing SOA systems over to a resource oriented architecture. I’m well past the ROA vs SOA debate (a ROA is a subset of SOA) but one lingering concern that I have about this and other discussions is this distinction between ROAs & WOAs.

I’m not quite sure that I can see a clear distinction between the two and I think having two TLAs for what may be the same underlying architecture type may be hindering understanding. Almost every core feature of WOA systems is derived from the underlying RESTful constraints. I’d imagine that if we could do away with one of these TLAs then the underlying concepts, and the differences from big-SOA, will become more accessible to all. (Note: I accept that there are non-RESTful anomalies out there mixed onto the www substrate but these edges always seem to fall by the wayside as the www has evolved.)

When I get chance I’ll try write up something more detailed on this but in the meantime if you have some clear examples of use cases where a WOA is a distinct superset of ROA then I’m sure they’d help my thinking.

Platforms like ESBs were built to solve the problem of interconnecting and integrating enterprise systems. The disconnect between how an ESB-based or RESTful solution approach the same problem can mostly be explained by the fact that they both evolved from very different starting points, and therefore have very different principals at their core. 

ESBs, having evolved from DCE-RPC, CORBA based RPC architectures, have always required static definition of non-uniform remote interfaces.  The modern WS-Deathstar-type ESBs are still based on this concept, with WSDL interface typically backed by statically compiled implementations (or if you are exotic XML based mediation languages).  The remote interface semantics can be augmented by a variety (cluster f&@k?) of ‘enterprisey’ features like transactions, routing and reliability defined by a collection of related specifications but it turns out that implementations based on some of those specifications do not interoperate or scale.  This is primarily a function of questionable specification development processes and over complexity.

In the end the non-uniform interfaces at the heart of these solutions tightly couple the clients and servers and this coupling is the underlying limiting factor on the adaptability and extensibility of these systems.  I still really struggle to find a successful real world internet-available IDL or WSDL based web service that has been used in a series of completely unexpected contexts (i.e. mashups) or that has evolved beyond more than one or two revisions without completely breaking backward compatibility for old clients.

In parallel the RESTful HTTP standard emerged and provided the platform for the Web that has scaled and adapted to truly global proportions.  This wasn’t an accident – the Web succeeded because of the RESTful principals upon it was built – stateless, client server, layered systems that support caching and of course the true use of Hypertext As The Engine Of Application State (thankfully now referable to as ‘the hypertext constraint‘).  To me HATEOAS is the most critical feature of RESTful solutions. It is the one feature that negates the need for non uniform interfaces and it is here where the two approaches diverge dramatically.  It also isn’t an easy concept to grasp for folks who are used to traditional RPC oriented systems. 

Regardless, the recent emergence of dynamic languages like Python and Ruby, and their respective web application frameworks, Django and Rails, is now making it economically efficient for anyone to produce RESTful web services that can scale and that are reusable.  Steve is just expressing his opinion that this approach works better.

Don’t get me wrong – the ESB approach has been proven to succeed and
scale but primarily in limited deployment environments and usually only when considerable resources are thrown at the solution.  To put it another way, the inherent limits in non-uniform interface based systems can be pushed through use of sophisticated tools and language bindings but this only be achievable when considerable resources are thrown at the solution.  I think this is what Steve is getting at – I don’t think Steve is saying ESBs are “bad”, just that they are not the best platform for building internet scale services/resources

(Hey, check it out, I didn’t mention SOA once!)

BTW, I’m reading The Future Of Ideas at the moment, hope to write more about that soon, but it does relate to the notion of control mentioned in Steve’s latest followup post.

“They’re simplifying something very complex — and something that’s made up of very distinct components. They need BPEL experts to work on the BPEL part, Java experts to work on the Java part, etc.,” Geyer said in an e-mail. “Trying to put people with such different skill sets and interests into one committee — and get quorums for meetings and approvals on drafts — would slow things down. I think we’ll see results much faster with six groups working in parallel.”

Six committees, working on interrelated specifications in parallel and not a reference implementation or compatability test suite in sight. This is bordering on the sublime but apparently a lot of people want this the SCA beast to live. Why exactly is beyond me.

Speaking of implementations, does anybody know what the heck really happened in the Tuscany project that caused the Fabric3 fork? I mean, it must have been pretty bad to fork a project that was backed by both IBM and BEA.

(via Tim Bray’s ongoing)

• How should RESTful clients bootstrap into a RESTful system, or rather, how does the client determine the correct payload to send to the first RESTful resource it interacts with?  Currently it appears that there must be some out-of-band definition of the first RESTful resource request that the client has ‘baked-in’ in order to initiate a conversation with a set of RESTful resources.  HTTP Web browsers solve this problem through (bookmarked) URLs.  Should all RESTful resource clients do the same?
• How should RESTful clients interpret arbritrary XML response payload?  The response XML is presumably defined by a schema (somewhere) but I’m wondering how is a RESTful client expected to interpret the semantics of the XML unless it has prior knowledge of the schema.  Is this prior knowledge just assumed? Again, HTML browsers succeed in this regard because they are build on the semantics of the HTML language schema.

I’d be very interested if anyone has insights into the above two questions.  If knowledge of the schema(s) is assumed (Aristotle’s post implies this) then perhaps all folks are looking for is guidance on how to determine which XML element definitions are expected/returned by RESTful resource request/reponses. 

Perhaps that’s where the value in WADL lies.  Unfortunately WADL seems to promote static resource set definitions, as WSDL does for WS-* service endpoints.  Perhaps if the WADL resource set definitions could also reference other WADL resource set definitions then we might have something a bit more dynamic?

Two excellent pieces serve to proffer incredible insight into the experiences of those who have hyper-succeeded in the past:

• Inside MySpace.com, by Baseline Magazine is a great read about how MySpace scaled their architecture from zero to over 26 million user accounts, serving over 40 billion pages a month (isn’t that figure just incredible!).
• Database War Stories is a series of posts by Tim O’Reilly, interviewing folks from Second Life, Memeorandum, Craigslist and more. (The rest of the posts are linked to at the bottom of the first post.)

One common theme in many of these stories: periodically these guys are faced with the stark reality that incremental improvements to existing infrastructure will not sustain the current business model. It is testimony to the folks in charge that they trust their geeks enough to bet the company repeatedly on new architectures.

It is a high-risk world and there are many that fall by the wayside but the rewards for the brave are there for all to see.

AccuRev supports open standards and SOA-based development tool interoperability across Microsoft Windows, Linux, and UNIX platforms.

(emphasis mine)
I have to admit, this piqued my interest (actually my initial reaction was ‘wtf?’ but I digress).  Searching to figure out what SOA-based development tool interoperability is, I am now guessing that this is a reference to the Eclipse ALF (incubating) project.  ALF is a CruiseControl type framework that integrates various commercial SCM repositories, build management, testing and code scanning tools etc. via a BPEL-based orchestrations that control the overall “Application Lifecycle Framework”.

Unfortunately, ALF appears to be getting no love from the big players, which is a bit of a shame as we could all do with something a bit more sophisticated than CruiseControl+Ant for integrating and controlling large scale builds.  A Web Services API integration is, I think, a valid way to tie these types of systems together.  

After reviewing the ALF demos, I am actually a little surprised that the entire scope of what ALF is trying to achieve is managed under the Eclipse.org umbrella.  I can see a place for an ALF tools project in Eclipse.org certainly but I am surprised that the whole ALF server-side architecture appears to be being designed and developed in the same project.  I really don’t really see the connection between Eclipse.org and the design or implementation of service interfaces like the ALF Event Manager WSDL.  Projects like STP and WTP for example work to integrate with open standards like SCA and JEE but ALF does not seem to have that type of clear seperation between tools and target runtime implementation. 

Perhaps I am missing something but a recent interesting discussion on the eclipse.technology.alf newsgroup (Subject: Upcoming validation release review) raised these and other related concerns about the scope and deliverables of this project seems to re-enforce this observation.  It will be interesting to see how this all pans out.

• Frequent run-time checking of only the part of the contract you care about
• Minimal surface area dependence on Web service
• Lowest possible distance between client data representation and server representation
• Tolerant, robust, federated data modification (incomplete updates must never break the application)
• Managing multiple data sources in a maintainable way.  Use MVC to knit them/update them.

How this differs from any other SOA, beyond being an incomplete subset of considerations for any enterprise architect to muse over, is very questionable.  Why we need another term for this set of patterns is questionable.   How these patterns qualify as pragmatic when others (such as defining an interface versioning strategy up front) are somehow not is questionable. 

On a deeper level, how well-defined these patterns are is also questionable.  For example, I find it difficult to reconcile the use of the MVC pattern on an enterprise architecture level.  Are we supposed to define model services and view services?  Where does the controller get implemented and do we have to provide view callback interfaces (for the model to update the view).  Do the view services have to maintain state?  Sorry, the granualarity and complexity doesn’t fit distributed services at all.
 
And I’m still not sure what the term Client in WOA/Client signifies.  I have to agree with Sam Ruby – articles like this signify that the term SOA has entered the realm of Newspeak.  WOA/Client seems to be bordering on the stuff that Dilbert cartoons are made of.  It may only serve to further disconnect and confuse the very architects who are struggling to embrace existing, concrete, service oriented architecture definitions.

The questions were quite high level and so the results are equally woolly.  One trend in the results is that the self-proclaimed experts seem to be quite sure of what they are doing but the ‘others’ are not.  Hmm, stating the obvious perhaps?  A far more interesting question, the degree to which the experts agree with one another, was not tackled.   El Reg seems to have a pretty well informed reader base so I’m sure there is something useful in there somewhere for someone. 

The report was sponsered by IBM yet it doesn’t any WebSphere offerings (or any other vendor’s product for that matter).  Poor value for money Big Blue!  (It doesn’t mention ESBs or SCA either).

• The OSGi/JSR#291/JSR#277 issue is still very much alive although some efforts are being made to try achieve some degree of compatibility by using a common subset of Manifest headers. However differences apparently already exist in the drafts (such as the version range specifier format used) It is pretty sad to see such open (or should I say JCP-closed) disregard for interoperability between two overlapping JSR. Both are due for delivery in Dolphin (Java 7), perhaps by then the whole Java platform will have been rendered unusable by the plethora of other overlapping JSRs and we (the users) will all have moved on to coding in Ruby.
  Actually, by then, it may not matter – OSGi seems to be gaining considerable traction. Tuesday afternoon had a whole afternoon of talks and discussion from Richard Hall (Felix), Peter Kriens (OSGi Alliance/aQute), Marcel Offermans (Luminis) and a round table discussion involving folks from Apache Maven, Felix, Harmony, Directory Server projects. It seems some efforts are being made to try get Apache Jakarta projects to OSGi-fy their Manifests (via capabilities built into Maven)
• Woden looks good. It’ll be the WSDL2.0 processor that all Java folks end up using (if you need to parse WSDL2.0 that is!). It replaces WSDL4J which is being put out to pasture (it must be, I logged a bug against it over a year ago and still no feedback!)
• Axis 2 apparently has WSDL2.0 HTTP binding support with some restrictions (messages must be “IRI style” compatible). I’m not sure if one even needs to declare a HTTP binding in the WSDL, it seems to be automatically available for all services that have SOAP endpoints.
• Apache Synapse looks nice, in a cute kind of way. I’m not sure I’d call it an ESB since it is really just a very thin routing framework that runs on top of Axis 2 – it doesn’t have any management interface per say and the overview might have been a bit forward looking with regard to the capabilities of the current implementation Synapse looks more like an endpoint intermediary that you one might use to implement specific tasks (XPath/RegExp based routing, binding conversion, logging, endpoint authentication & authorization, transformation (XSLT, E4X, POJO based). I’m not sure you’d host your service implementations on it.
• Apache Tuscany work is ongoing and I bet it will be for some time yet! I’m not certain of their claim that it will simplify the development of business solutions, if only because it is based on SCA which is turning into an absolute beast of a set of specifications – think “one spec to rule them all” type big, and with big specifications comes complexity, not simplification. From an implementation point of view, a huge problem is that the SCA specifications do not have either a reference implementation nor a compatibility test suite and according to Simon Nash and Jeremy Boynes (both of IBM) there are no current plans to develop either. Also it seems curious that the SCA specifications are not being developed under the auspices of an open body like OASIS, W3C or the OMG – why not? Some would say the overall approach seems incredibly vulnerable to repeat the mistakes of CORBA. It will also be interesting to see if any convincing response is given to Ron Ten-Hove’s recent critique of SCA.

• Daryl Plummer stoked the fire with his Web Services at a Crossroads piece.
• Tim Bray suggests “Web Style” is a better moniker than “REST”. He also suggests the label “Web Service” should be taken out back and shot (haha).
• Joe McKendrick agrees that the SOA hype has gotten out of hand.
• Dion Hinchcliffe is still trying the align the two approaches and references yet-another-Gartner-acronym, “WOA” – Web Oriented Architecture.
• Tim Bray finally sums it all up by reiterating that he thinks this really is very important.

Thats a lot of clever folks trying to figure out how WOA will fit into the enterprise. It does seem likely to happen given past trend of internet technologies migrating into enterprises but at this early stage nobody seems to have a clear idea of exactly how, where and when.
I’m not suggesting that it will lead to the death of WS-*, too many large organizations have invested too much in WS-* for it to go away any time soon.

It is also interesting that nobody has yet mentioned how early WOA adapters will end up with a JBOWS architecture instead of a WOA, as has happened to some early SOA adapters. WOA doesn’t seem to offer anything to help alleviate this tendancy – if anything it may well suffer from it even more.
They also need to figure out how WOA will fit into developers and IT departments hands – they won’t get far without better development and management tooling.

Don Box wrote a while back asking for suggestions on how to split a hypothetical $100 budget to best improve the Microsoft HTTP/XML/REST platform. That Don hasn’t yet provided his own solution to the problem suggest that a) it isn’t as easy as he thought or b) MS don’t want to give too much advice away to their competitors Either way, there’s an opportunity for some clever Eclipse folks to get their thinking caps on and produce a WOA tools platform, maybe targetting the huge LAMP server platform.

My guess is, in the time honoured tradition of software engineering, someone will produce a new “architecture” layer that sits on top of both SOA and WOA, and over a long period or time one of the architectures will capture the magical point-of-no-return mindshare. Maybe the SaaS (Software as a Service) banner will evolve into that role?