“Metric_fu is a set of rake tasks that make it easy to generate metrics reports. It uses Saikuro, Flog, Flay, Rcov, Reek, Roodi, Churn, RailsBestPractices, Subversion, Git, and Rails built-in stats task to create a series of reports. It’s designed to integrate easily with CruiseControl.rb by placing files in the Custom Build Artifacts folder.” It’s really a pretty sweet tool. It generates all these reports and you can either ignore them or act on them. Not saying which of those we do… but that’s not important right now. If you’ve used metric_fu you probably ran into the NaN error. Good ol’ “Not A Number”. Generally the output looks something like this:
NaN
/usr/lib64/ruby/gems/1.8/gems/activesupport-2.3.4/lib/active_support/core_ext/float/rounding.rb:19:in round\_without\_precision'
/usr/lib64/ruby/gems/1.8/gems/activesupport-2.3.4/lib/active\_support/core\_ext/float/rounding.rb:19:inround’
/usr/lib64/ruby/gems/1.8/gems/metric_fu-1.3.0/lib/base/generator.rb:135:in round\_to_tenths'
/usr/lib64/ruby/gems/1.8/gems/metric\_fu-1.3.0/lib/generators/rcov.rb:85:into_h’
/usr/lib64/ruby/gems/1.8/gems/metric_fu-1.3.0/lib/base/generator.rb:131:in generate\_report'
/usr/lib64/ruby/gems/1.8/gems/metric\_fu-1.3.0/lib/base/generator.rb:53:ingenerate_report’
/usr/lib64/ruby/gems/1.8/gems/metric_fu-1.3.0/lib/base/report.rb:54:in add'
/usr/lib64/ruby/gems/1.8/gems/metric\_fu-1.3.0/lib/../tasks/metric\_fu.rake:6
/usr/lib64/ruby/gems/1.8/gems/metric\_fu-1.3.0/lib/../tasks/metric\_fu.rake:6:ineach’
/usr/lib64/ruby/gems/1.8/gems/metric_fu-1.3.0/lib/../tasks/metric_fu.rake:6
This particular error has cropped up several times for us, each time we do the same google searches, look at the same metric_fu sources, and slowly dissect which tests are causing this problem. Hopefully this post will help expedite debugging this error for you and your friends. The NaN error is the result of an Infinity/Infinity calculation in metric_fu. How Infinity gets in there in the first place is left as a reader exercise. Ultimately the problem resides in the fact that the scratch/rcov/rcov.txt output file does not contain the expected output. In fact, in our case, it contained a failing test. At this point, running rcov by itself (outside of metric_fu) would result in expected output - everything worked and there were no failing tests! So we ran metric_fu again with all tests disabled except for rcov (by editing our rake task). Metric_fu failed in the exact same place. Great - at least now it is narrowed down (even if it does take a few minutes to run). Taking a look at the failing test nothing stood out, until we dug a little deeper. This test class was overriding a cattr_reader with a cattr_accessor so it could change the value for testing. No big deal, it worked fine when run by itself …. Turns out that this wasn’t the only test class that was modifying the class variable. Metric_fu loads up all of the tests (functionals & units) and runs them together. We had one unit test class that was modifying a class variable (and not setting it back!) and a functional test class that was making assertion based on the expected default value of that class variable. We had indirectly made our tests order dependent. The reason we never saw it in our normal tests was that our functional and unit tests run separately (via rake). Summary:
- Look in scratch/rcov/rcov.txt to find a failing test
- Look for usage of class variables or constants - namely the changing of them (and not setting them back).
Our Solution:
Our solution was to only temporarily change the class variable, and only do it when we need to:
def temporarily_set_observer_limit_to(count, &block)
old_limit = MyClass.observer_limit
MyClass.observer_limit=count
yield
MyClass.observer_limit=old_limit
end
A better solution would be to get rid of the class variables… Another solution would be to change metric_fu to handle a failing test more gracefully In the end, we are left with this rule:
Don’t change class variables in tests!
… or better yet …
Don’t use class variables!
Friends don’t let friends use class variables. Partnership for cattr-free coding.