Doing battle with metric_fu

Metric_fu is a set of rake tasks that make it easy to generate metrics reports. It uses Saikuro, Flog, Flay, Rcov, Reek, Roodi, Churn, RailsBestPractices, Subversion, Git, and Rails built-in stats task to create a series of reports. It’s designed to integrate easily with CruiseControl.rb by placing files in the Custom Build Artifacts folder.” It’s really a pretty sweet tool. It generates all these reports and you can either ignore them or act on them. Not saying which of those we do… but that’s not important right now. If you’ve used metric_fu you probably ran into the NaN error. Good ol’ “Not A Number”. Generally the output looks something like this:

NaN
/usr/lib64/ruby/gems/1.8/gems/activesupport-2.3.4/lib/active_support/core_ext/float/rounding.rb:19:in round\_without\_precision' /usr/lib64/ruby/gems/1.8/gems/activesupport-2.3.4/lib/active\_support/core\_ext/float/rounding.rb:19:inround’
/usr/lib64/ruby/gems/1.8/gems/metric_fu-1.3.0/lib/base/generator.rb:135:in round\_to_tenths' /usr/lib64/ruby/gems/1.8/gems/metric\_fu-1.3.0/lib/generators/rcov.rb:85:into_h’
/usr/lib64/ruby/gems/1.8/gems/metric_fu-1.3.0/lib/base/generator.rb:131:in generate\_report' /usr/lib64/ruby/gems/1.8/gems/metric\_fu-1.3.0/lib/base/generator.rb:53:ingenerate_report’
/usr/lib64/ruby/gems/1.8/gems/metric_fu-1.3.0/lib/base/report.rb:54:in add' /usr/lib64/ruby/gems/1.8/gems/metric\_fu-1.3.0/lib/../tasks/metric\_fu.rake:6 /usr/lib64/ruby/gems/1.8/gems/metric\_fu-1.3.0/lib/../tasks/metric\_fu.rake:6:ineach’
/usr/lib64/ruby/gems/1.8/gems/metric_fu-1.3.0/lib/../tasks/metric_fu.rake:6

This particular error has cropped up several times for us, each time we do the same google searches, look at the same metric_fu sources, and slowly dissect which tests are causing this problem. Hopefully this post will help expedite debugging this error for you and your friends. The NaN error is the result of an Infinity/Infinity calculation in metric_fu. How Infinity gets in there in the first place is left as a reader exercise. Ultimately the problem resides in the fact that the scratch/rcov/rcov.txt output file does not contain the expected output. In fact, in our case, it contained a failing test. At this point, running rcov by itself (outside of metric_fu) would result in expected output - everything worked and there were no failing tests! So we ran metric_fu again with all tests disabled except for rcov (by editing our rake task). Metric_fu failed in the exact same place. Great - at least now it is narrowed down (even if it does take a few minutes to run). Taking a look at the failing test nothing stood out, until we dug a little deeper. This test class was overriding a cattr_reader with a cattr_accessor so it could change the value for testing. No big deal, it worked fine when run by itself …. Turns out that this wasn’t the only test class that was modifying the class variable. Metric_fu loads up all of the tests (functionals & units) and runs them together. We had one unit test class that was modifying a class variable (and not setting it back!) and a functional test class that was making assertion based on the expected default value of that class variable. We had indirectly made our tests order dependent. The reason we never saw it in our normal tests was that our functional and unit tests run separately (via rake). Summary:

  1. Look in scratch/rcov/rcov.txt to find a failing test
  2. Look for usage of class variables or constants - namely the changing of them (and not setting them back).

Our Solution:

Our solution was to only temporarily change the class variable, and only do it when we need to:

def temporarily_set_observer_limit_to(count, &block)
old_limit = MyClass.observer_limit
MyClass.observer_limit=count
yield
MyClass.observer_limit=old_limit
end

A better solution would be to get rid of the class variables… Another solution would be to change metric_fu to handle a failing test more gracefully In the end, we are left with this rule:

Don’t change class variables in tests!

… or better yet …

Don’t use class variables!

Friends don’t let friends use class variables. Partnership for cattr-free coding.

Compleat Rubyist - Day 1

Day 1 of 2 of my notes for the Compleat Rubyist training course

Ruby Versions and Implementations - David

http://ruby-versions.net/ - David’s home for ruby versions & implementations for learning & historical reference Ruby version manager - http://rvm.beginrescueend.com - lets you install several ruby versions/implementations and easily switch between (including your own custom compiled version). Suggestion - don’t install as root, even though it is allowed. Notes on a few of the existing options:

  • MacRuby - interacts with Cocoa
  • Rubinius - Ruby in Ruby
  • JRuby - Ruby on JVM
  • REE - optimized - created by Phusion Passenger team
  • MagLev - built in object persistence, repository instead of files, smalltalk-ish
  • IronRuby - Ruby on .NET
  • URABE - ?

rvm allows you to compare performance between versions/implementation:

rvm ruby-1.8.6,ruby1.9.2 benchmark filename.rb

Why does everyone use 1.8 instead of 1.9?

  • Same amount of people are using it as last year (like almost no one)
  • Rails considerations
  • 1.9 is not 100% backwards compatible
  • 1.8.7 backported many of the features of 1.9, so people feel safer

Ruby Enterprise Edition has major memory and speed improvements Highlights of changes between 1.8 & 1.9

  • Enumerators
  • Method parameters
  • Block variable binding & scope
  • Syntax changes

Discussion

  • “Ruby 1.9 was plenty different enough to be 2.0” - David
  • 1.9.1 is currently the stable supported version and has been for about a year.
  • 1.8.6 to 1.8.7 was a big jump - major backporting of 1.9 features into 1.8
  • 1.8.7 was a safe harbor for those that wanted 1.9 features but were scared of 1.9
  • rails3 + ruby 1.9.1 = segfaults :( … works with certain revisions of 1.9.2 HEAD

Enumerator object - 1.9

  • mixes in Enumerable
  • you write the ‘each’ method
    • borrow from another object (can be lazy)
    • or
    • pass in code block on instantiation (via a yielder)
  • once it knows how to ‘each’, it can do select, map, each_cons ….
  • 1.8.6 - require ‘enumerator’ - Enumerable::Enumerator
  • 1.9.x - no require needed - promoted to top level ‘Enumerator’

side bar: each_cons vs each_slice Note to self: I need to memorize each/select/map/collect What methods do I get with Enumerator that I don’t get with Enumerable (Array)?

Enumerator.instance_methods - Array.instance_methods
:with_index, :with_object, :next, :rewind

Method argument semantics required args can now come after optional args def m(a, b=1, c) def m(a, b, c) def m((a,b),c) required arguments get filled first *Block variable scope**

  • probably the most important/annoying/significant change
  • breaks stuff in surprising ways
  • but if it does break stuff, you were likely doing something buggy before

Example 1:

a = [1,2,3]
a.each {|x| p x}

x gets value of 3 when you are done |x| literally assigns x ( |x=…| ), so it becomes available outside the block Example 2:

a = 1
array.each { a = 2 }

does not change the value of a

  • Matz said he wished he had done this from the beginning
  • Unifies the parameter syntax between methods & lambas

1.9 Miscellany

  • no more String#each
    • “Hello”[0] = “H” #in 1.8 it returns 72
  • new instance_exec is like instance_eval but takes a param to inject

JRuby - ask David about using ArrayList instead of [] in playpoker.rb

The Testing Landscape

Test::Unit is gone in 1.9 Test::Unit::TestCase -> Mini::Test::TestCase Mini::Test has a new option: refute_match require ‘shoulda’ - makes it more like RSpec without going full RSpec RSpec is the defacto standard for Behavior-driven testing Jeremy - wrote ‘context’ and ‘match’ Given/When/Then - cucumber is most popular “Think in units of features rather than units of code” - Gregory require ‘could’ - another tool - include feature test right in the same file as tests Test data

  • YAML/CSV - hard to maintain, csv is kinda nice because you can open in spreadsheet program (or rather your testers can)
  • model_stubbing http://github.com/technoweenie/model_stubbing
  • Factories is the new hotness
  • mocking/stubbing
    • can extend a class to do it
    • can use OpenStruct to do it require ‘ostrich’
    • Jeremy likes using ‘rr’
    • RSpec has it’s own stubbing
    • flexmock
  • Proxies
    • Proxies are like mocks & stubs & real code combined
    • Proxies are the Ken Jennings of mocks & stubs