It’s really a pretty sweet tool. It generates all these reports and you can either ignore them or act on them. Not saying which of those we do… but that’s not important right now.

If you’ve used metric_fu you probably ran into the NaN error. Good ol’ “Not A Number”.
Generally the output looks something like this:

This particular error has cropped up several times for us, each time we do the same google searches, look at the same metric_fu sources, and slowly dissect which tests are causing this problem. Hopefully this post will help expedite debugging this error for you and your friends.

The NaN error is the result of an Infinity/Infinity calculation in metric_fu. How Infinity gets in there in the first place is left as a reader exercise. Ultimately the problem resides in the fact that the scratch/rcov/rcov.txt output file does not contain the expected output. In fact, in our case, it contained a failing test.

At this point, running rcov by itself (outside of metric_fu) would result in expected output – everything worked and there were no failing tests!

So we ran metric_fu again with all tests disabled except for rcov (by editing our rake task).
Metric_fu failed in the exact same place. Great – at least now it is narrowed down (even if it does take a few minutes to run).

Taking a look at the failing test nothing stood out, until we dug a little deeper.
This test class was overriding a cattr_reader with a cattr_accessor so it could change the value for testing.
No big deal, it worked fine when run by itself ….

Turns out that this wasn’t the only test class that was modifying the class variable.
Metric_fu loads up all of the tests (functionals & units) and runs them together.
We had one unit test class that was modifying a class variable (and not setting it back!) and
a functional test class that was making assertion based on the expected default value of that class variable.
We had indirectly made our tests order dependent.

The reason we never saw it in our normal tests was that our functional and unit tests run separately (via rake).

Summary:

1. Look in scratch/rcov/rcov.txt to find a failing test
2. Look for usage of class variables or constants – namely the changing of them (and not setting them back).

Our Solution:

Our solution was to only temporarily change the class variable, and only do it when we need to:

A better solution would be to get rid of the class variables…
Another solution would be to change metric_fu to handle a failing test more gracefully

In the end, we are left with this rule:

Don’t change class variables in tests!

… or better yet …

Don’t use class variables!

Friends don’t let friends use class variables.
Partnership for cattr-free coding.

Ruby Versions and Implementations  – David

http://ruby-versions.net/ – David’s home for ruby versions & implementations for learning & historical reference

Ruby version manager – http://rvm.beginrescueend.com – lets you install several ruby versions/implementations and easily switch between (including your own custom compiled version). Suggestion – don’t install as root, even though it is allowed.

Notes on a few of the existing options:

• MacRuby – interacts with Cocoa
• Rubinius – Ruby in Ruby
• JRuby – Ruby on JVM
• REE – optimized – created by Phusion Passenger team
• MagLev – built in object persistence, repository instead of files, smalltalk-ish
• IronRuby – Ruby on .NET
• URABE – ?

rvm allows you to compare performance between versions/implementation:

rvm ruby-1.8.6,ruby1.9.2 benchmark filename.rb

Why does everyone use 1.8 instead of 1.9?

• Same amount of people are using it as last year (like almost no one)
• Rails considerations
• 1.9 is not 100% backwards compatible
• 1.8.7 backported many of the features of 1.9, so people feel safer

Ruby Enterprise Edition has major memory and speed improvements

Highlights of changes between 1.8 & 1.9

• Enumerators
• Method parameters
• Block variable binding & scope
• Syntax changes

Discussion

• “Ruby 1.9 was plenty different enough to be 2.0″ – David
• 1.9.1 is currently the stable supported version and has been for about a year.
• 1.8.6 to 1.8.7 was a big jump – major backporting of 1.9 features into 1.8
• 1.8.7 was a safe harbor for those that wanted 1.9 features but were scared of 1.9
• rails3 + ruby 1.9.1 = segfaults    … works with certain revisions of 1.9.2 HEAD

Enumerator object – 1.9

• mixes in Enumerable
• you write the ‘each’ method
  • borrow from another object (can be lazy)
  • or
  • pass in code block on instantiation (via a yielder)
• once it knows how to ‘each’, it can do select, map, each_cons ….
• 1.8.6 – require ‘enumerator’ – Enumerable::Enumerator
• 1.9.x – no require needed – promoted to top level ‘Enumerator’

side bar: each_cons vs each_slice

Note to self: I need to memorize each/select/map/collect

What methods do I get with Enumerator that I don’t get with Enumerable (Array)?

Enumerator.instance_methods - Array.instance_methods
:with_index, :with_object, :next, :rewind

Method argument semantics
required args can now come after optional args
def m(a, b=1, c)
def m(a, *b, c)
def m((a,b),c)
required arguments get filled first

Block variable scope

• probably the most important/annoying/significant change
• breaks stuff in surprising ways
• but if it does break stuff, you were likely doing something buggy before

Example 1:

a = [1,2,3]
a.each {|x| p x}

x gets value of 3 when you are done

|x| literally assigns x ( |x=…| ), so it becomes available outside the block

Example 2:

a = 1
array.each { a = 2 }

does not change the value of a

• Matz said he wished he had done this from the beginning
• Unifies the parameter syntax between methods & lambas

1.9 Miscellany

• no more String#each
  • “Hello”[0] = “H”  #in 1.8 it returns 72
• new instance_exec is like instance_eval but takes a param to inject

JRuby – ask David about using ArrayList instead of [] in playpoker.rb

The Testing Landscape

Test::Unit is gone in 1.9
Test::Unit::TestCase -> Mini::Test::TestCase
Mini::Test has a new option: refute_match
require ‘shoulda’ – makes it more like RSpec without going full RSpec
RSpec is the defacto standard for Behavior-driven testing

Jeremy – wrote ‘context’ and ‘match’
Given/When/Then – cucumber is most popular
“Think in units of features rather than units of code” – Gregory
require ‘could’ – another tool – include feature test right in the same file as tests

Test data

• YAML/CSV – hard to maintain, csv is kinda nice because you can open in spreadsheet program (or rather your testers can)
• model_stubbing  http://github.com/technoweenie/model_stubbing
• Factories is the new hotness
  • FactoryGirl
  • Machinist – http://github.com/notahat/machinist
    • create shams & blueprints
    • “way slower” than fixtures
• mocking/stubbing
  • can extend a class to do it
  • can use OpenStruct to do it require ‘ostrich’
  • Jeremy likes using ‘rr’
  • RSpec has it’s own stubbing
  • flexmock
• Proxies
  • Proxies are like mocks & stubs & real code combined
  • Proxies are the Ken Jennings of mocks & stubs

We’ve uncovered a slew of things to work on. The first obvious place was to have the static content served by Apache rather than the mongrels. This has been on the todo-list for a long time, but has always fallen to the wayside. We knocked that off and looked towards the other items that were slowing us down. That’s when I uncovered some TCPSocket weirdness within memcache-client 1.7.4 that comes with activesupport 2.3.5.

During some tests we noticed a severe lag which we narrowed down to the fact that we were pointing to a list of memcached servers that contained one that didn’t exist. Turns out that when the memcached hostname resolves, but is not pingable, memcache-client 1.7.4 waits 3 seconds before responding with an error message (in addition it doesn’t mark the server as “down”, which I think is also a bug). This 3 second delay happens on RHEL 4, and in some brief tests on Ubuntu 10.9, it was even worse, taking over 30 seconds to respond. My guess is that there is some OS level setting that affects this, but I have yet to locate it. The fun part however, is that this problem does not exist in our old environment where we run acivesupport 2.1.2 which uses memcache-client 1.5.0.

Turns out, in 1.5.0, the memcache-client uses a 250ms timeout when calling TCPSocket.new. Something that was lost on the way to 1.7.4. Some initial tests of simply adding this CONNECT_TIMEOUT back in have been promising. It’s currently not throwing the right exception or marking the server as “down”, but once I do that, I will see about posting the source somewhere.