August 22nd, 2007
- Pre Google, YouTube had 2 sysadmins, 2 scalability software engineers, 2 feature developers, 2 network enginers and 1 DBA. Yowza! talk about a productive team!
- Python is fast enough. In their small team, development speed was as important as execution speed. I’m confident the same applies for Ruby on Rails too (this was a primary consideration for our choosing RoR instead of Java for our new nooked platform).
- MySQL replication works, to the point when replication of writes starts to significantly impact reads from the replicas. Once you hit that point start working at night to develop an alternative middle tier that relies on partitioning the data into multiple shards. You work on the alternative middle tier at night because you’ll spend all day trying to keep the replicas up to speed with the master. Map incoming requests to the correct shard by doing a user id lookup on a dedicated table (in a heavily cached local database) to redirect requests to the correct shard.
Speaking of MapReduce, I see that Ruby has an implementation – Starfish. Cool.