11 January 2012

A better way of testing Rails application with minitest.


In this post I would like to share with you some of my work, that I have just done. And it is about a better way of testing Rails application with minitest.
In my previous post you can find my quick tutorial about how it is possible to use minitest with Rails. But following that post you can also find some inconvenience, with all that manual work around, that needs to be done. That was not good enough for me, so I moved the stuff forward into more cleaner solution.

The major improvements have been made for minitest-rails gem. Most of the modules inclusions have been moved from test helper into gem's lib.
The integration test generator with templates has been added. Models, controllers and integration tests should now work just fine, without any special care about stuff like subject, @controller, etc. to be explicitly defined. Also some Rails specific assertions (with new defined spec expectations) are now available with no effort.
Also, thanks to capybara_minitest_spec gem, it is now possible to use capybara within integration tests, with some extra spec expectations provided.
Everything now should be clean and simple.

While using either Test::Unit or RSpec, I usually used to get some help from shoulda-matchers gem. I would like to use those handy matchers with minitest, as well. I found  minitest-matchers gem makes this to be possible.
Using both gems together I was able to save tons of LOC again. But to have that, I needed few extra steps. To make it all to be available in one go, I have created gem called minitest-rails-shoulda.

Please note, that all my work on those gems is a kind of "not necessary get to know how it works, but just make it work" and it may need some code improvements. Also, it has not been merged into upstream repositories yet. That means those new features are not included in currently released gems. That is why I am using my forks from github in this post. This may change in the future and if so, expect this post to be updated. But until then, remember - you are using this stuff on your own risk;)

OK, let me show you at last how I test Rails application using minitest.
Just to let you know, in case you want to follow this post, that I was using ruby 1.9.3p0 and Rails 3.1.3 to go through.

1. Application Setup

First. Create a brand new rails application.
$ rails new TestApp -T
Option -T skips test unit as we don't want to use it.

Add to our Gemfile
group :test, :development do
  gem 'minitest-rails',
      :git => "git@github.com/rawongithub/minitest-rails.git",
      :branch => "gemspec"
end
group :test do
  gem 'minitest-rails-shoulda',
      :git => "git@github.com/rawongithub/minitest-rails-shoulda.git"  
  gem 'capybara_minitest_spec'
end
This should match the following dependent gems to be installed automatically:
  • minitest
  • minitest-rails
  • minitest-matchers
  • shoulda-matchers
Don't forget to run bundler against our new Gemfile content:
$ bundle install
Note: You may need to run this either with --without assets option or add gem 'therubyracer' into assets group of your Gemfile.

Run the minitest installation generator:
$ rails generate mini_test:install
This should create minitest_helper.rb file within test directory

Edit our test/minitest_helper.rb to look like this:
require "minitest/autorun"
require "minitest/rails"
require "minitest/pride" # let's have awesome colorful output;)

ENV["RAILS_ENV"] = "test"
require File.expand_path('../../config/environment', __FILE__)

class MiniTest::Rails::Spec
  # Add methods to be used by all specs here
end

class MiniTest::Rails::Model
  # make fixtures available within models spec
  include MiniTest::Rails::Fixtures
end

class MiniTest::Rails::Controller
  # Add methods to be used by controller specs here
end

class MiniTest::Rails::Helper
  # Add methods to be used by helper specs here
end

class MiniTest::Rails::Mailer
  # Add methods to be used by mailer specs here
end

class MiniTest::Rails::Integration
  # Add methods to be used by integration specs here
end

Make minitest generators to use spec syntax and fixtures. Add this to our config/application.rb:
config.generators do |g|
  g.test_framework :mini_test, :spec => true, :fixture => true
  g.integration_tool :mini_test
end

2. Testing Models

Generate a sample model:
$ rails generate model User name:string
This should create files:
test/fixtures/users.yml
test/models/user_test.rb
db/migrate/[timestamp]_create_users.rb

Create database unless exists, run migrations and prepare database for test environment:
$ rake db:create
$ rake db:migrate
$ rake db:test:prepare

Edit test/fixtures/users.yml with sample data:
tester:
  name: "Tester"

Put some sample tests into test/models/user_test.rb:
require "minitest_helper"
describe User do
  fixtures :users
  let(:tester) { users(:tester) }
  context "database" do
    it { must have_db_column(:name).of_type(:string) }
  end
  context "instance methods" do
    it "#name returns name" do
      tester.name.must_equal "Tester"
    end
  end
end

And here we are. We can now run our model tests:
$ rake test:models

3. Testing Controllers

Generate sample controller and view:
$ rails generate controller Website index
This should create some files. We need one of them only for now.
Edit test/controllers/website_controller_test.rb with sample test:
require "minitest_helper"
describe WebsiteController do
  context "index action" do
    before do
      get :index
    end
    it { must_respond_with :success }
    it "must render index view" do
      must_render_template :index
    end
  end
end

Run controllers tests and watch how they pass.
$ rake test:controllers

4. Integration Tests

Generate an integration test
$ rails generate integration_test Website
This should create test/integration/website_test.rb

Edit test/integration/website_test.rb with sample test:
require "minitest_helper"
describe "Website Integration Test" do
  context "website#index" do
    before do
      get "/website/index"
    end
    it { must_respond_with :success }
    #capybara usage example
    it "website index page must be accessible" do
      visit website_index_path
      page.must_have_content("Website#index")
    end
  end
end
Note: To make it work we need to keep naming convention for #describe argument:
"<Name> Integration Test"

Finally, run integration tests.
$ rake test:integration

5. Summary

As you can see testing Rails application with minitest can be very straightforward with minimum effort.
Although there is a lot of work to be done yet, what we have so far should be enough to start using this lightweight and modern testing framework with Rails.
Thanks for reading, and please give me a feedback.

24 comments:

  1. Great post—doing the integration and controller tests as specs in minitest has been giving me fits. Thanks!

    ReplyDelete
  2. Agreed. I'm surprised there's not more talk & examples of people switching to MiniTest.

    Functional specs have been driving me crazy as well. Most recently, I get errors that @routes is nil, though I've included Rails.application.routes.url_helpers.

    I'll check out the new additions to minitest-rails for some clues. Thanks for all your work, Rafał.

    ReplyDelete
  3. I just cloned https://github.com/blowmage/minitest-rails and I don't see generators or templates for integration tests in there.

    Am I missing something?

    ReplyDelete
    Replies
    1. @Jesse Clark
      As it is explained in the post - I am using 'gemspec' branch of my fork of mninitest-rails git@github.com:rawongithub/minitest-rails.git

      Delete
  4. Great article, but surprised that you're promoting fixtures!

    ReplyDelete
    Replies
    1. I am surprised as well;)
      But in fact fixtures are not so evil. They work well for some projects. They usually give better testing performance. And I don't always find fixtures maintenance much harder than factories.

      Delete
  5. This is locked into Rails 3.1 have you tried it with Rails 3.2?

    ReplyDelete
    Replies
    1. Rails 3.1 was the current version, while I was making this post. No, I didn't test that against Rails 3.2.

      Delete
    2. Good news. This is now unlocked for Rails 3.2

      Delete
  6. This is very helpful, thank you so much! However, I am unable to nest describe/context blocks like your examples show. I get "NameError: wrong constant name" as constantize is called on the string following context. Any ideas?

    ReplyDelete
    Replies
    1. Thanks for your feedback. This issue occurs with Rails 3.2 and has been just fixed. Look into github for details.

      Delete
    2. I'm on Rails 3.2.8 and still encounter this issue. Any idea where or when the fix went in?

      Delete
  7. Thank you for putting this together. When I followed your instructions on a Rails 3.2.2 app, I get this:


    $ bundle install
    Fetching git@github.com:rawongithub/minitest-rails.git
    Permission denied (publickey).
    fatal: The remote end hung up unexpectedly
    Git error: command `git clone 'git@github.com:rawongithub/minitest-rails.git' "/Users/peter/.rvm/gems/ruby-1.9.3-p0@webtools/cache/bundler/git/minitest-rails-e6e7e5209b3dae8f38869f8a79718ce8bce01938" --bare --no-hardlinks` in directory /Users/peter/gdctools/webtools has failed.

    Looks like I have no permission to clone the Git repo.

    ReplyDelete
  8. Hi Rafal, I figured out the problem. Your blog says:

    gem 'minitest-rails',
    :git => "git@github.com:rawongithub/minitest-rails.git",
    :branch => "gemspec"

    But it requires SSH keys and proper access. For everyone else, they should use:

    gem 'minitest-rails',
    :git => "git://github.com/rawongithub/minitest-rails.git",
    :branch => "gemspec"


    Thank you for sharing.

    ReplyDelete
    Replies
    1. Good catch. Fixed. Thanks for your feedback.

      Delete
  9. Thanks for sharing this. Just wondering if you have any success to get Controller spec working with Ruby 1.9.3+ Rails 3.2.3, I bumped to this https://github.com/blowmage/minitest-rails/issues/37.

    ReplyDelete
  10. I must be missing something... when I try either the controller or integration test above, my environment continues to report "NoMethodError: undefined method `must_respond_with'...

    My Gemfile includes the 'minitest-rails' and the 'minitest-rails-shoulda', and my test_helper.rb includes the [require 'minitest/rails'] statement.

    What's missing to get must_respond_with to work?

    ReplyDelete
  11. Also, I'm finding that the 'get' call in both the example integration and controller tests above are giving me this error:

    NoMethodError: private method `get' called for nil:NilClass

    Can anyone tell me what I'm missing?

    ReplyDelete
  12. Okay, I've answered my own question about the method_missing error for "must_repond_with". It turns out the example code in this article isn't correct.

    it { must_respond_with :success }
    should be written as
    it { must respond_with :success }

    ReplyDelete
  13. Rafal, your intro states "Models, controllers and integration tests should now work just fine, without any special care about stuff like subject, @controller, etc. to be explicitly defined."

    I'm having trouble, still, with your examples for controller and integration tests. Written exactly at you have them, I get

    NameError: undefined local variable or method `subject' for #<#:0x000001033ed100>
    /Users/tgriffin/.rvm/gems/ruby-1.9.3-p194/gems/actionpack-3.1.4/lib/action_dispatch/testing/assertions/routing.rb:175:in `method_missing'

    How would I go about setting or defining 'subject' in a controller or integration test?

    Thanks,
    Tim

    ReplyDelete
    Replies
    1. I think you can explicitly define subject by using #subject(&block) method.

      Delete
  14. This is a nice post, but your test files are missing the classes. See test/models/user_test.rb and test/controllers/website_controller_test.rb and test/integration/website_test.rb. You have require and describe... but no classes.. for example, class FooControllerTest < ActionController::TestCase

    ReplyDelete