The Brian Olore Story

Building an API with Rails Panel  – http://en.oreilly.com/rails2010/public/schedule/detail/14502

Panelists: Joe Ferris (thoughtbot, inc), Jeremy Kemper (37signals), Marcel Molina (Twitter), Rick Olson (GitHub), Derek Willis (The New York Times)

Authentication

• OAuth 1.0 spec is difficult to implement – signature difficulties, etc
• OAuth 2 uses ssl & tokens – much simpler to implement, has overhead of SSL

Input & Response Formats

• Hoptoad now only supports XML – because they are write heavy and use streaming they were able to simplify their interface and make it faster by dropping support for other formats
• NYTimes – XML & JSON, expanded formats based on user needs
• Twitter – XML, JSON, RSS, Atom – probably going to move towards JSON only to scale up – XML is too verbose – lots of memcache full of XML fragments
• 37signals – uses RJS

To Version or Not to Version

• Twitter – hard problem – fork with multiple deploys? Inheritance? – lots of work. What do we need versioning for? Lotsa stuff we thought needed versioning actually didn’t. Looking for tips on how to do easily.
• 37signals – is it really a problem?  If you think it needs a new versions, just make it a new resource
• How do you support legacy – 37signals: Given enough lead time application developers can deal with it fairly quickly (1 month).
• Hoptoad – no matter how many people you tell, they won’t upgrade. You won’t loose as may people as you think. People don’t upgrade until something is broken.
• Twitter – explosion of resources could be a problem. Implement new option/param/header to allow people to get new functionality during transition period. Look at analytics & how much time has passed then take out conditional and drop previous. “It’s going to break if you don’t upgrade.”
• NYTimes – “we’re shutting this off in 3 months”

Scaling

• NYTimes uses mashery and varnish
• 37signals – reverse proxy caching, usage patterns of API are usually very different than patterns of web traffic
• Twitter – cache not only fragments, but fragments of fragments – so that the more volatile stuff can expire without losing the more static data. Consider a streaming API instead of having users continuously poll. (mentioned “Cache Money” which I had to google cuz it’s a sweet name: http://github.com/nkallen/cache-money/tree/master)
• Github – take advantage of HTTP headers & caching

Code Separation

• Should there be a separation between your web & api code?
• Github – API code was in same code as controllers – became separate API controller – looking at moving to a Sinatra app
• Twitter – used to be DRY and works great for most applications. web/mobile/api encapsulated in respond to’s. Different code for rate limits, etc, “became unruly”, merge conflicts when split across teams, etc. Now they duplicate, but it’s not that bad because of skinny controllers & fat models.
• 37signals – just because twitter does it, doesn’t mean you have to. Much much easier to keep API in same code, should be exception to separate – this is why respond_to exists.

Security Concerns

• Twitter – “You’re going to get screwed no matter how awesome you are. And it’s a really hard problem.”
• Github: Don’t use cookies & sessions in your API requests
• 37signals – don’t worry about authenticating (use oauth) worry about authorization

Developer Communications

• Twitter – Tools & documentation – Page for “Is there a problem with the system?” – portal for developers
• Hoptoad – make sure your documentation is correct & up to date!
• 37signals – Don’t just put up a mailing list.
• NYTimes – use your own API – build samples & actually use/run them
• Github – documentation is actually stored in git
• 37signals – dogfooding

Tags: api, railsconf

Mike Clark http://en.oreilly.com/rails2010/public/schedule/detail/14136

This is going to be mostly an iPhone application development session, not Rails.
Demo app is performs CRUD operations on a Rails 3 site from an iPhone app

http://github.com/clarkware/iphone-rails-tutorial

Basic Railsy JSON controller methods

Standard iPhone callback methods: applicationWillTerminate, didFinishLaunchingWithOptions

in .h file – create instance variables in the @interface and add:
@property(nonatomic, retain) NSMutableArray *goals;

in .m use: @synthesize goals (initializes instance variable), also release in dealloc()

viewDidLoad – called when the view is loaded
reference “goals” with “self.goals” to access variable through property

“Do not fear the square brackets”

cellForRowAtIndexPath – don’t delete what’s there (that’s what makes scrolling through large list fast), just add your own logic to the end of the method, before “return cell”;

commitEditingStyle – delete from our array – allows deletion in app. Restart app to get data back (since it’s in memory)

StringWithContentsOfURL – fetch URL synchronously (blocks main UI thread)
json-framework is a strict JSON parser and generator
return NSDictionary or NSArray of NSDictionaries

refreshButton – “action:@selector(refresh)” — is like using “send” in ruby. target:self” means that self.refresh will be called on click

networkActivityIndicatorVisible – network spinner

NSURLRequest, NSURLConnection, sendSynchronousRequest

Shift – Apple – R brings up debug console in Xcode

After break – Asynchronous Networking, Authentication, Error handling ….

Async Networking – done with the Delegate pattern – http://en.wikipedia.org/wiki/Delegation_pattern

Use an OperationQueue to background work – NSOperationQueue, NSInvocationOperation – don’t use NSThread
Always update the UI on the main thread – performSelectorMainThread

HTTPRiot is a simple REST library that makes it easy to interact with RESTful resources (inspired by httparty) – responses converted to NSDictionary and NSArray objects.

http://allseeing-i.com/ASIHTTPRequest/ – flexible, low-level library with tons of features – File uploads, post form-encoded data, basic authentication, progress indictors, etc. Not REST specific, doesn’t handle json/xml by default, but very useful for low-level stuff

ObjectiveResource is an Objective-C port of ActiveResource – Automatically serializes/deserializes objects (to/from JSON or XML) – Assumes Rails RESTful conventions – adds methods to NSObject

didFinishLaunchWithOptions – only gets called once – when app starts up

ObjectiveResource needs the following to give json a default root object:
ActiveRecord::Base.include_root_in_json = true

Tags: iphone, railsconf

Jim Weirich – @jimweirich

If for some crazy reason you don’t already have git installed, go download it from here – http://git-scm.com/download

Source Control Made Easy – Pragmatic Bookshelf Screen Cast

git config –global user.name “Brian Olore”
git config –global user.email “brian@olore.net”
git config –global core.autocrlf input

git init – creates a new, empty repository

Git is about changes, not files.

Always additive – Information is never destroyed – makes it really really hard to screw up your repository

SHA1 used for preventing duplication – all files referenced using their hash
- Is there additional meta-data that has things like executable flag, etc ??

Tags point to snapshots & never move

Branches point to snapshots and move with commits
Can branch after committing changes
Cheap, local branches

git clone <url> – make an exact copy of remote working copy
git pull <url> – receive changes from remote url into your working copy

Shared remote archive – allows sharing between many users – like svn, but if remote goes away, can still push/pull between users without the centralized repository

git log –pretty=oneline
5e870f0441fb9a30ad2711e186cb46a1cafd9e30 pulling from command line
63537c6c788c4893cf928509339f84cb9f243c6b First commit

git log –pretty=oneline –abbrev-commit
5e870f0 pulling from command line
63537c6 First commit

git add – adds the changes to be committed, not the file. Can have multiple adds

git diff – compare un-staged (things not yet “add”ed)
git diff –cached – compare staged area (things “add”ed) to “commit”ted

Both gitx (for Macs) and gitk (any platform) are useful in exploring log history.

git reset – changes a branch pointer

git reset –hard – changes the branch pointer and checks out into your working directory.

git rebase master – replays branch changes (rewrites commits) as if they happened on master

DO NOT rebase shared branches – only reset/rebase on local repositories

http:/nvie.com/git-model – “A successful Git branching model”

Tags: git, railsconf

This past week or so I’ve been concentrating on improving the performance of our system. Thankfully this isn’t a solo task and my teammate really knows what he’s doing. Luckily I was able to contribute based on my past experiences with things like cache settings in Apache, ulimit settings and other tweaks here and there.

We’ve uncovered a slew of things to work on. The first obvious place was to have the static content served by Apache rather than the mongrels. This has been on the todo-list for a long time, but has always fallen to the wayside. We knocked that off and looked towards the other items that were slowing us down. That’s when I uncovered some TCPSocket weirdness within memcache-client 1.7.4 that comes with activesupport 2.3.5.

During some tests we noticed a severe lag which we narrowed down to the fact that we were pointing to a list of memcached servers that contained one that didn’t exist. Turns out that when the memcached hostname resolves, but is not pingable, memcache-client 1.7.4 waits 3 seconds before responding with an error message (in addition it doesn’t mark the server as “down”, which I think is also a bug). This 3 second delay happens on RHEL 4, and in some brief tests on Ubuntu 10.9, it was even worse, taking over 30 seconds to respond. My guess is that there is some OS level setting that affects this, but I have yet to locate it. The fun part however, is that this problem does not exist in our old environment where we run acivesupport 2.1.2 which uses memcache-client 1.5.0.

Turns out, in 1.5.0, the memcache-client uses a 250ms timeout when calling TCPSocket.new. Something that was lost on the way to 1.7.4. Some initial tests of simply adding this CONNECT_TIMEOUT back in have been promising. It’s currently not throwing the right exception or marking the server as “down”, but once I do that, I will see about posting the source somewhere.

Tags: performance, ruby, timeout

One of the things I really like about my new job is pair programming. On Tuesday of this week, there was a bit of a panic as we tried to figure out how we were going to get work done with everyone in different locations due to the impending snow storm on Wednesday. To me, it was almost comical. I mean, I had been working from home for the last 6 or so years. I never had someone looking over my shoulder as I wrote code. One day wasn’t going to be a big deal… was it? Given the way this team has worked for the last few years, there really was no plan for how to operate when we aren’t all in the same room.

There was talk of iChat and sharing screens and some other not-so-simple-to-setup environment stuff, but in the end we coordinated what we were working on via chat and agreed we’d pair up on Thursday to review changes/updates we made or wanted to make. Even though I ended up doing the same on Thursday (no plow on my street all day, I couldn’t leave the house!), I was able to be productive and reviewed my work today with some team members.

It was weird working from home again. It was definitely different – I had to use my home laptop and my office space was a bit of a wreck. It took a little while to get totally setup with the VPN etc, but I ended up cranking out some good work.

My final conclusion is that I really like pairing, and working in an open space. Maybe I have great teammates, maybe I’m still in the honeymoon phase of pair programming, who knows, but for now, I’m loving it.

Tags: pair programming, snow, team