redis is not use for everything because entire dataset used to have to fit into RAM – would have to shard across machines

github – uses redis for routing, can add, remove.modify routes quickly & easily – can determine what server to put new users on, etc

Old – HAProxy sending to mongrels
New – Unicorn – instead of receiving requests, it asks for one using ‘select’, pooled, forks similar to passenger

Used 6 or 7 different bg – SQS, DelayedJob
Used Delayed job for 2 years

resque is built on redis, jobs are serialized to json and workers can be in any language http://github.com/defunkt/resque

resque has a plugin API – 15 or so plugins already exist – http://wiki.github.com/defunkt/resque/plugins

has admin interface

github has 35 types of background jobs

Notes: lightning fast talker! Lots of good information though. I’d have a beer with this guy!