<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>Aehso's Output &#187; mapreduce</title>
	<atom:link href="http://www.xlml.com/aehso/category/mapreduce/feed/" rel="self" type="application/rss+xml" />
	<link>http://www.xlml.com/aehso</link>
	<description>John O'Shea's musings, observations and opinions on anything and everything.</description>
	<lastBuildDate>Mon, 25 Jan 2010 13:31:40 +0000</lastBuildDate>
	<generator>http://wordpress.org/?v=2.9.1</generator>
	<language>en</language>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
			<item>
		<title>Seattle Conference on Scalability : YouTube&#8217;s Growing Pains.</title>
		<link>http://www.xlml.com/aehso/2007/08/22/seattle-conference-on-scalability-youtubes-growing-pains/</link>
		<comments>http://www.xlml.com/aehso/2007/08/22/seattle-conference-on-scalability-youtubes-growing-pains/#comments</comments>
		<pubDate>Wed, 22 Aug 2007 17:36:42 +0000</pubDate>
		<dc:creator>aehso</dc:creator>
				<category><![CDATA[google]]></category>
		<category><![CDATA[mapreduce]]></category>
		<category><![CDATA[mysql]]></category>
		<category><![CDATA[python]]></category>
		<category><![CDATA[rails]]></category>
		<category><![CDATA[ruby]]></category>
		<category><![CDATA[scalability]]></category>
		<category><![CDATA[youtube]]></category>

		<guid isPermaLink="false">http://www.xlml.com/aehso/2007/08/22/seattle-conference-on-scalability-youtubes-growing-pains/</guid>
		<description><![CDATA[Cuong Do, one of the original YouTube engineers, gave a great talk (50+ minutes but well worth it) at Google&#8217;s recent Seattle Conference on Scalability.&#160; Some takeway thoughts:

Pre Google, YouTube had 2 sysadmins, 2 scalability software engineers, 2 feature developers, 2 network enginers and 1 DBA.&#160; Yowza! talk about a productive team!
Python is fast enough.&#160; [...]]]></description>
			<content:encoded><![CDATA[<p>Cuong Do, one of the original YouTube engineers, <a href="http://video.google.com/videoplay?docid=-6304964351441328559">gave a great talk</a> (50+ minutes but well worth it) at Google&#8217;s recent <a href="http://www.google.com/events/scalability_seattle/">Seattle Conference on Scalability</a>.&nbsp; Some takeway thoughts:</p>
<ul>
<li>Pre Google, YouTube had 2 sysadmins, 2 scalability software engineers, 2 feature developers, 2 network enginers and 1 DBA.&nbsp; Yowza! talk about a productive team!</li>
<li>Python is fast <em>enough</em>.&nbsp; In their small team, development speed was as important as execution speed.&nbsp; I&#8217;m confident the same applies for Ruby on Rails too (this was a primary consideration for our choosing RoR instead of Java for our new nooked platform).</li>
<li>MySQL replication works, to the point when replication of writes starts to significantly impact reads from the replicas.&nbsp; Once you hit that point start working at night to develop an alternative middle tier that relies on partitioning the data into multiple shards.&nbsp; You work on the alternative middle tier at night because you&#8217;ll spend all day trying to keep the replicas up to speed with the master. Map incoming requests to the correct shard by doing a user id lookup on a dedicated table (in a heavily cached local database) to redirect requests to the correct shard.
</li>
</ul>
<p>Of course they now get to play with all of Google&#8217;s in-house toys (<a href="http://labs.google.com/papers/mapreduce.html">MapReduce</a>, <a href="http://labs.google.com/papers/gfs.html">GFS</a> etc) so doubt if he gives another presentation next year it&#8217;ll probably be a very different one.</p>
<p>Speaking of MapReduce, I see that Ruby has an implementation &#8211; <a href="http://www.rufy.com/starfish/doc/">Starfish</a>.&nbsp; Cool.</p>
]]></content:encoded>
			<wfw:commentRss>http://www.xlml.com/aehso/2007/08/22/seattle-conference-on-scalability-youtubes-growing-pains/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
	</channel>
</rss>

