RailsConf – Day 3 – Million Dollar Mongo

June 9th, 2010 by Brian filed in work has no comments

Million Dollar Mongo - Obie Fernandez (Hashrocket), Durran Jordan (Hashrocket)
mongo was a good choice because only one patient record was worked on at a time

get entire object graph in a single query
previously did 35 joins and took 20 minutes to return

comparison between mongo, couch & cassandra
big selling point was dynamic queries – nice happy medium when coming from relational databases
“I don’t want to write any more ****** XML, so goodbye Cassandra”

No universal agreement that a NOSQL solution was the right idea

“Easiest database to install ever (except maybe Redis)”
No schema migrations

Forked MongoMapper -> Monoid
- MongoMapper was very young

Mongoid
- Prefers Embedding Documents
- Atomic Operations by Default, based on attribute changes
- rich criteria api
- master/slave suppor
- simple versioning
- is NOT ActiveRecord (and should be treated as such)
- Features always have performance in mind

mongo – can index in background as of 1.4
regex queries on fields
Denormalization is acceptable, embed as much as possible (no FK constraints)

Hybrid DB model
- Hierarchical Data Stored in MongoDB
- Relational Data stored in MySQL
- Data that required transactions stored in MySQL
- Simple Data in Redis
- Favor suitability over simplicity

It’s ok to use more than one datastore

Working without transactions
- Atomic updates generally cover you
- fairly easy to roll your own optimistic locking
- need to do manual rollback in integration tests
- selenium tests can run in a separate process

in production
- deployed on engine yard cloud / ec2
- 0.5 TB of data
- put mongodb on it’s own utility space
- feed it lots of ram, lots of disk space
- chef scripts easy due to simple installation

Tags: ,

RailsConf – Day 2 – MongoDB Metamorphosis

June 8th, 2010 by Brian filed in work has no comments

The MongoDB Metamorphosis: Data as DocumentsKyle Banker (10gen)

http://try.mongodb.com – mini tutorial

MongoDB tries to fit between key-value stores & relational databases

Data as Documents – documents are in BSON (binary json)

mongoDB operates on documents using special syntax

update() takes a query and an update (what to do) .. the update is applied to the results of the query

Embedded documents – i.e. Comments become embedded within Posts
Or you can normalize – Post id within each comment
Decisions to be made on what makes most sense for how you need to access the data

Embedded – fast queries, document always appears with parent

Hash is the object used to represent MongoDB objects

MongoMapper

Very informational, I want to try using this!

Tags: ,

RailsConf – Day 2 – Beyond git push heroku

June 8th, 2010 by Brian filed in work has no comments

Beyond Git Push Heroku: Battle Stories from Cloud SamuraisOren Teich (Heroku), Morten Bagai (Heroku)
heroic had down time this morning, so we’ll talk about that for a bit then resume our normal schedule
use something similar to nagios
problem was occurring somewhere in http stack
form hypothesis & find a way to test it quickly and in a non-disruptive manner – don’t want to make the problem worse
disabled misbehaving systems – took them out of the cluster – didn’t turn off, wanted to diagnose

2 of the 3 cofounders here in room
Do one thing well – deploy a rails application onto the web
$git push heroku master
let developers use tools to deploy that they are already using
“No step 1″

heroku stacks – let’s you target a version of a vm & libraries that you want to target – linux version, ruby, etc

http://docs.heroku.com/rails3

http://www.scvngr.com

http://www.syphir.com

http://getcloudapp.com

pandastream – cloud based video encoding

Anything can be a cloud service as long as it’s deployed in the same infrastructure – memcached, monogohq, etc

Tags: ,

RailsConf – Day 2 – What should we work on next?

June 8th, 2010 by Brian filed in work has no comments

What should we work on next? – Brian Doll (New Relic)

My note: this is probably a sales pitch on RPM, but at least I know that gong in :)

  • Performance is value
  • Apdex – how your customers feel about the service they are getting – how satisfied they are
  • Awesome “Knuth is my homeboy” image with appropriate quote: “premature optimization is the root of all evil”
  • Nice charts to compare metrics vs 24 hours ago, a week ago, a month ago

My note: I was right, and now I am bored…but I am going to go read about solr

Tags:

RailsConf – Day 2 – Metrics Magic

June 8th, 2010 by Brian filed in work has no comments

Metrics Magic - Aaron Bedra (Relevance, Inc.) – maintains rcov

How did we get here?
Technical Debt – but there is no bailout program! Good idea if you intend on paying back.

Metrics are a great way to facilitation conversation
It’s ok to ask others how/why the code was written

if its hard to write tests around your code, that’s a smell

output of metrics tools is just text – you can make these metrics tool fail the build !

There’s no substitute for peer review

rcov can fail the build by setting “–failure-threshold n”
- should increase the threshold as your code coverage goes up so that you know when it drops

reek – fail the build by capturing the output
result = %x{reek app lib}
exit 1 unless result.empty?

flog -
flog appĀ  awk ‘NR==4′ | awk ‘{print $1}’
exit 1 if result.gsub(“:”,”").to_i >= 20

metric-fu
wraps tests in pretty pictures

Compare charts to previous sprints – generate conversation

Great presentation materials – cartoony, fun
Good speaker, useful demo

Tags: ,