Aehso’s Output

Is it a Ghetto or will it Rule?  Fairly diverse opinions (understatement!) but I think I’ll side with Tim on this one

Wonderful post (and followup) by Steve Vinoski, spawning lots of other discussions.  I should be surprised by some of the comments but… I’m not - I suspect agendas, pride and reputations may be at play.  Imho, and I stress humble opinion, his views are completely valid. Can enterprise architects disregard the architecture, constraints and protocols that helped the Web scale and adapt to be the global platform it is today?  If they want to build something that will adapt and scale then I also think the answer has to be No.

Platforms like ESBs were built to solve the problem of interconnecting and integrating enterprise systems. The disconnect between how an ESB-based or RESTful solution approach the same problem can mostly be explained by the fact that they both evolved from very different starting points, and therefore have very different principals at their core. 

ESBs, having evolved from DCE-RPC, CORBA based RPC architectures, have always required static definition of non-uniform remote interfaces.  The modern WS-Deathstar-type ESBs are still based on this concept, with WSDL interface typically backed by statically compiled implementations (or if you are exotic XML based mediation languages).  The remote interface semantics can be augmented by a variety (cluster f&@k?) of ‘enterprisey’ features like transactions, routing and reliability defined by a collection of related specifications but it turns out that implementations based on some of those specifications do not interoperate or scale.  This is primarily a function of questionable specification development processes and over complexity.

In the end the non-uniform interfaces at the heart of these solutions tightly couple the clients and servers and this coupling is the underlying limiting factor on the adaptability and extensibility of these systems.  I still really struggle to find a successful real world internet-available IDL or WSDL based web service that has been used in a series of completely unexpected contexts (i.e. mashups) or that has evolved beyond more than one or two revisions without completely breaking backward compatibility for old clients.

In parallel the RESTful HTTP standard emerged and provided the platform for the Web that has scaled and adapted to truly global proportions.  This wasn’t an accident - the Web succeeded because of the RESTful principals upon it was built - stateless, client server, layered systems that support caching and of course the true use of Hypertext As The Engine Of Application State (thankfully now referable to as ‘the hypertext constraint‘).  To me HATEOAS is the most critical feature of RESTful solutions. It is the one feature that negates the need for non uniform interfaces and it is here where the two approaches diverge dramatically.  It also isn’t an easy concept to grasp for folks who are used to traditional RPC oriented systems. 

Regardless, the recent emergence of dynamic languages like Python and Ruby, and their respective web application frameworks, Django and Rails, is now making it economically efficient for anyone to produce RESTful web services that can scale and that are reusable.  Steve is just expressing his opinion that this approach works better.

Don’t get me wrong - the ESB approach has been proven to succeed and
scale but primarily in limited deployment environments and usually only when considerable resources are thrown at the solution.  To put it another way, the inherent limits in non-uniform interface based systems can be pushed through use of sophisticated tools and language bindings but this only be achievable when considerable resources are thrown at the solution.  I think this is what Steve is getting at - I don’t think Steve is saying ESBs are “bad”, just that they are not the best platform for building internet scale services/resources

(Hey, check it out, I didn’t mention SOA once!)

BTW, I’m reading The Future Of Ideas at the moment, hope to write more about that soon, but it does relate to the notion of control mentioned in Steve’s latest followup post.

[Update: Some of the presentation files are now available on the RailsConf Europe website]
Slight delay on writing this one up as I was in transit on Thurs and at a wedding on Friday.  Anyway, this summary is going to be a lot shorter as I didn’t get as much out of Wednesday’s sessions.

Rails Hydra: Synthesizing an Application out of Multiple Rails Codebases (Craig R. McClanahan, Nick Sieger, Sun Microsystems)

Good talk on building services using several Rails applications, the Sun guys also got a chance to demo use of NetBeans 6 for Rails development (with some live demo debugging thrown in along with great audience participation!) Most of the demo centered around making it easier to develop DRY ActiveResource implementations. ActiveResource is almost definitely the right underpinnings for any RESTful service implementation but I think it still needs a bit more work around the edges.

Using a HAXOR Approach for Peace and Productivity (Tim Dysinger)

Good talk on how to manage interaction between designers and developers when developing Rails applications that use HTML, Ajax and XML Over REST. Quite high level (and if I’m honest, I was working away in the background so I couldn’t give it my full attention…)

Browser-based Testing of Massive Ajax-using Rails Applications with Selenium (Till Vollmer)
Good overview of using Selenium to test web applications from within the browser, covering use of Selenium IDE (browser based Javascript IDE), Selenium RC and the Selenium on Rails Plugin. Highlighted that Selenium struggles a bit with testing Ajax heavy web pages but it is possible with hand crafted scripts that use waitForVisible and/or waitForElementPresent events…

Functional JavaScript Development with Prototype (Ben Nolan)
Bit of an edge talk for the JavaScript fanatics but Ben presented well and it seemed well received by the audience…

Ruby on Rails leads you to the e-business (Quentin Tousart)
Mildly interesting talk on experiences gained in building two e-commerce websites using Rails.

Obscure Data Formats, Workflow, and Remote Synchronization (Chad Thatcher)
Another interesting case study on building a Rails front-end for a legacy data format (in this case the RISM format used by the British Library to catalog music manuscripts). Interesting use of composed_of in the Rails model objects to compensate for the fact that the underlying data was in hierarchical rather than relational form.

That’s about it, I met up with Sean Hanley from exoftware and David Rice . Mental note to self, get into RubyIreland…

[Update: Some of the presentation files are now available on the RailsConf Europe website]
Some quick notes from some of the excellent conference sessions that I attended yesterday at RailsConf Europe.  I wasn’t here for the Tutorials on Monday though now I wish I had been - the quality of the presentations (at least the ones I’m picking) is excellent.  Aside, Berlin is a cool spot, I must come back here sometime again to have a decent look around…

Deployment and Continuous Integration from the Trenches, (Fernand Galiana - LiquidRail)

• All about latest developments for Capistrano 2.0 - it seems to have matured considerably in recent months.
• Use multistage_ext
• Use shared project capfiles to keep things DRY
• Use remote repository cache to speed up deployments
• Read Jamis Buck’s blog and the Capistrano Google Group

Really Scaling Rails, (Britt Selvitelle - Twitter)
I’ve seen presentations on Twitter scalability, and even since then they have had a few more high profile outages (at least high profile amongst Twitter users). A couple of interesting takes from this talk:

• Twitter uses Apache, mod_proxy_blancer and mongrel servers
• All user traffic is handled by a single MySQL server. That server does have a slave that can be promoted to master (for redundancy). They have a couple of other MySQL databases for use internally for reporting etc.
• They set mongrel’s num_procs to 1. This means that each mongrel server instance will not queue requests from mod_proxy_balancer - they will only accept one at a time. The side effect is if the concurrent request count > mongrel server count then users start getting error pages. Strangely, they’d rather users got errors than risking loosing queued requests whenever they have to restart a mongrel instance (mongrel apparently waits only 60s before sending itself a TERM signal when shutting down).

Rails Full Text Search with Ferret

A little different to what I was expecting, Ferret provides an indexing service for arbitrary strings (documents), similar to Apache Lucene in many respects.  Worth a look, if you need to support full text search within your rails application.

Tabnav: Do We Really Need a Plugin for Tabbed Navigation? (Paola Dona - Seesaw)

The title of this one was a a bit modest as Paola presented a slew of new Rails Widgets that SeeSaw have developed, all of which seem to be very flexible. Widgets for include tabbed nav (or course), site nav, tables (blocks), nubbins, show/hide blocks, help popups were all demoed. Their widgets integrate very nicely into the rails views/templates - all in all, it looks at least good enough for use in creating rails app prototypes and it appears they might be flexible enough to be embedded into a production application…
The Rest of REST (Roy T. Fielding) - slides

Good historical view of where Roy came from and how the principles of REST have always been such an important underpinning of the IETF’s thinking behind standardization of key web specifications like HTTP, URI and HTML. He provided a good overview of how the web architectural style is defined as a set of constraints:

• client server
• stateless
• caching
• uniform interface
• layered systems
• code-on-demand

There was some interesting discussion on what is missing from Rails though Roy’s first two suggestions drew comments that he might have missed features in Rails that do what he wanted.  His last suggestion, for Rails to guide the developer into using hypertext as the engine for of application state (man they really have to find a shorter name for that!) was a fair comment - seems like an incredibly difficult problem to solve in a generic framework like Rails but as he said, it’d be a first…
Last talk was Craig McClanahan finishing off the day with a short Rails and the Next Generation Web pitch.  Now I hadn’t realised Craig had switched from Java to Ruby development and he seems to be loving it.  Craigs name has all over the Apache code I’ve worked with for years now - he was one of the original Tomcat Catalina & Struts developers and he also co-authored the Servlet and JSF specifications.  He had an interesting anecdote about how the Struts developers all suddenly found themselves working on non-struts based projects…

I’m heading to RailsConf Europe 2007 in the morning for a few days of Rails goodness. If you fancy a meetup (any other Irish devs attending?) do drop me a comment or email me…

Cuong Do, one of the original YouTube engineers, gave a great talk (50+ minutes but well worth it) at Google’s recent Seattle Conference on Scalability.  Some takeway thoughts:

• Pre Google, YouTube had 2 sysadmins, 2 scalability software engineers, 2 feature developers, 2 network enginers and 1 DBA.  Yowza! talk about a productive team!
• Python is fast enough.  In their small team, development speed was as important as execution speed.  I’m confident the same applies for Ruby on Rails too (this was a primary consideration for our choosing RoR instead of Java for our new nooked platform).
• MySQL replication works, to the point when replication of writes starts to significantly impact reads from the replicas.  Once you hit that point start working at night to develop an alternative middle tier that relies on partitioning the data into multiple shards.  You work on the alternative middle tier at night because you’ll spend all day trying to keep the replicas up to speed with the master. Map incoming requests to the correct shard by doing a user id lookup on a dedicated table (in a heavily cached local database) to redirect requests to the correct shard.

Of course they now get to play with all of Google’s in-house toys (MapReduce, GFS etc) so doubt if he gives another presentation next year it’ll probably be a very different one.

Speaking of MapReduce, I see that Ruby has an implementation - Starfish.  Cool.

Bill de hÓra has a nice piece on Struts 1 that touches on something about MVC that I’ve never quite liked in the web application context:

In pattern language terms, the MVC pattern has been reapplied to the Web domain without consideration for the forces induced by the Web. MVC frameworks tend to be inwardly focused contraptions of server sided redirects. Alternatives: On the web, a suitable pattern is View, Model, Template. A request to a URL is dispatched to a View. This View calls into the Model, performs manipulations and prepares data for output. The data is passed to a Template that is rendered an emitted as a response. ideally in web frameworks, the controller is hidden from view.

I like the View, Model, Template pattern suggestion, the separation of View (which data can be displayed) and Template (how it gets displayed) is an important one that I think too often gets lost in web application development.

If only the framework could be the entire controller all on its own though! Even the Ruby On Rails (the poster boy web application framework of the moment) cannot do everything unaided.  Writing controllers is not too bad if you are exposing RESTful resources - much custom controller logic is then reusable though things like authentication still intrude.  RPC oriented web appications are the ones that can really end up suffering from horrendously complex controller definitions.

The other main difference between Java/Struts1 and something like RoR is the considerable benefit gained by being able to define the controller completely within in the host language rather then having to resort to a code+XML definition.  I don’t think this should be underestimated. Personally I have always found the biggest impediment to grokking, testing and maintaining Java/Struts1 application code was the mixed Java/XML implementation, and the IDE support it almost demands when used in anger.

BTW, big contratulations to Bill on getting the APP spec (nearly) finished!

(JM, you’ll love this one)

Peter Thomas has an interesting stack diagram of a Sprint/Hibernate web application in action. No, honestly, there is no recursion in there.
Maybe, just maybe, this is why Ruby on Rails is grabbing so much mindshare amonst Java developers these days. If you havn’t see the RoR “Create a weblog in 15 minutes screencast” yet, do not pass go, do not collect €200.

(via ongong)