Aehso’s Output

RailsConf Europe Day 1

[Update: Some of the presentation files are now available on the RailsConf Europe website]
Some quick notes from some of the excellent conference sessions that I attended yesterday at RailsConf Europe.  I wasn’t here for the Tutorials on Monday though now I wish I had been - the quality of the presentations (at least the ones I’m picking) is excellent.  Aside, Berlin is a cool spot, I must come back here sometime again to have a decent look around…

Deployment and Continuous Integration from the Trenches, (Fernand Galiana - LiquidRail)

• All about latest developments for Capistrano 2.0 - it seems to have matured considerably in recent months.
• Use multistage_ext
• Use shared project capfiles to keep things DRY
• Use remote repository cache to speed up deployments
• Read Jamis Buck’s blog and the Capistrano Google Group

Really Scaling Rails, (Britt Selvitelle - Twitter)
I’ve seen presentations on Twitter scalability, and even since then they have had a few more high profile outages (at least high profile amongst Twitter users). A couple of interesting takes from this talk:

• Twitter uses Apache, mod_proxy_blancer and mongrel servers
• All user traffic is handled by a single MySQL server. That server does have a slave that can be promoted to master (for redundancy). They have a couple of other MySQL databases for use internally for reporting etc.
• They set mongrel’s num_procs to 1. This means that each mongrel server instance will not queue requests from mod_proxy_balancer - they will only accept one at a time. The side effect is if the concurrent request count > mongrel server count then users start getting error pages. Strangely, they’d rather users got errors than risking loosing queued requests whenever they have to restart a mongrel instance (mongrel apparently waits only 60s before sending itself a TERM signal when shutting down).

Rails Full Text Search with Ferret

A little different to what I was expecting, Ferret provides an indexing service for arbitrary strings (documents), similar to Apache Lucene in many respects.  Worth a look, if you need to support full text search within your rails application.

Tabnav: Do We Really Need a Plugin for Tabbed Navigation? (Paola Dona - Seesaw)

The title of this one was a a bit modest as Paola presented a slew of new Rails Widgets that SeeSaw have developed, all of which seem to be very flexible. Widgets for include tabbed nav (or course), site nav, tables (blocks), nubbins, show/hide blocks, help popups were all demoed. Their widgets integrate very nicely into the rails views/templates - all in all, it looks at least good enough for use in creating rails app prototypes and it appears they might be flexible enough to be embedded into a production application…
The Rest of REST (Roy T. Fielding) - slides

Good historical view of where Roy came from and how the principles of REST have always been such an important underpinning of the IETF’s thinking behind standardization of key web specifications like HTTP, URI and HTML. He provided a good overview of how the web architectural style is defined as a set of constraints:

• client server
• stateless
• caching
• uniform interface
• layered systems
• code-on-demand

There was some interesting discussion on what is missing from Rails though Roy’s first two suggestions drew comments that he might have missed features in Rails that do what he wanted.  His last suggestion, for Rails to guide the developer into using hypertext as the engine for of application state (man they really have to find a shorter name for that!) was a fair comment - seems like an incredibly difficult problem to solve in a generic framework like Rails but as he said, it’d be a first…
Last talk was Craig McClanahan finishing off the day with a short Rails and the Next Generation Web pitch.  Now I hadn’t realised Craig had switched from Java to Ruby development and he seems to be loving it.  Craigs name has all over the Apache code I’ve worked with for years now - he was one of the original Tomcat Catalina & Struts developers and he also co-authored the Servlet and JSF specifications.  He had an interesting anecdote about how the Struts developers all suddenly found themselves working on non-struts based projects…

Off to RailsConf Europe

I’m heading to RailsConf Europe 2007 in the morning for a few days of Rails goodness. If you fancy a meetup (any other Irish devs attending?) do drop me a comment or email me…

Seattle Conference on Scalability : YouTube’s Growing Pains.

Cuong Do, one of the original YouTube engineers, gave a great talk (50+ minutes but well worth it) at Google’s recent Seattle Conference on Scalability.  Some takeway thoughts:

• Pre Google, YouTube had 2 sysadmins, 2 scalability software engineers, 2 feature developers, 2 network enginers and 1 DBA.  Yowza! talk about a productive team!
• Python is fast enough.  In their small team, development speed was as important as execution speed.  I’m confident the same applies for Ruby on Rails too (this was a primary consideration for our choosing RoR instead of Java for our new nooked platform).
• MySQL replication works, to the point when replication of writes starts to significantly impact reads from the replicas.  Once you hit that point start working at night to develop an alternative middle tier that relies on partitioning the data into multiple shards.  You work on the alternative middle tier at night because you’ll spend all day trying to keep the replicas up to speed with the master. Map incoming requests to the correct shard by doing a user id lookup on a dedicated table (in a heavily cached local database) to redirect requests to the correct shard.

Of course they now get to play with all of Google’s in-house toys (MapReduce, GFS etc) so doubt if he gives another presentation next year it’ll probably be a very different one.

Speaking of MapReduce, I see that Ruby has an implementation - Starfish.  Cool.

Model/View/Template or Model/View/Controller?

Bill de hÓra has a nice piece on Struts 1 that touches on something about MVC that I’ve never quite liked in the web application context:

In pattern language terms, the MVC pattern has been reapplied to the Web domain without consideration for the forces induced by the Web. MVC frameworks tend to be inwardly focused contraptions of server sided redirects. Alternatives: On the web, a suitable pattern is View, Model, Template. A request to a URL is dispatched to a View. This View calls into the Model, performs manipulations and prepares data for output. The data is passed to a Template that is rendered an emitted as a response. ideally in web frameworks, the controller is hidden from view.

I like the View, Model, Template pattern suggestion, the separation of View (which data can be displayed) and Template (how it gets displayed) is an important one that I think too often gets lost in web application development.

If only the framework could be the entire controller all on its own though! Even the Ruby On Rails (the poster boy web application framework of the moment) cannot do everything unaided.  Writing controllers is not too bad if you are exposing RESTful resources - much custom controller logic is then reusable though things like authentication still intrude.  RPC oriented web appications are the ones that can really end up suffering from horrendously complex controller definitions.

The other main difference between Java/Struts1 and something like RoR is the considerable benefit gained by being able to define the controller completely within in the host language rather then having to resort to a code+XML definition.  I don’t think this should be underestimated. Personally I have always found the biggest impediment to grokking, testing and maintaining Java/Struts1 application code was the mixed Java/XML implementation, and the IDE support it almost demands when used in anger.

BTW, big contratulations to Bill on getting the APP spec (nearly) finished!