Nik Cubrilovic over at TechCrunch has a post up about why Twitter is having a hard time scaling:
Twitter is unique in that it needs to parse a large number of messages and deliver them to multiple recipients, with each user having unique connection.
Hmm, that sounds like something familiar… some other site I know.. somethi… Oh, wait… EMAIL.
Why doesn’t Twitter just set up a little mini internet-like SMTP-inspired network behind their load balancer, with the equivalent of SMTP servers handling a handful of twitter users on each server, put a bunch of routers in between, and do it that way? I mean, isn’t Twitter just email with a bunch of functionality cut out? Isn’t posting on twitter the same thing as sending an one-line email to a bunch of friends? And doesn’t everyone just read their tweets in the equivalent of a featureless inbox?
Forget about the whole one big database table idea that we think is mandatory for web apps. Just copy a distributed architecture that you know will work and will scale.