How to properly setup RSpec

This post is recommended for everyone from total beginners to people who literally created RSpec.

Starting a new project

When you start a new ruby project, it's common to begin with:

$ git init
$ rspec --init

to create a repository and some sensible TDD structure in it.

Or for rails projects:

$ rails new my-app -T
$ cd my-app

Then edit Gemfile adding rspec-rails to the right group:

group :development, :test do
  gem "rspec-rails"

end

And:

$ bundle install
$ bundle exec rails g rspec:install

I feel all those Rails steps really ought to be folded into a single operation. There's no reason why rails new can't take options for a bunch of popular packages like rspec, and there's no reason why we can't have some kind of bundle add-development-dependency rspec-rails to manage simple Gemfile automatically (like npm already does).

But this post is not about any of that.

What test frameworks are for

So why do we even use test frameworks really, instead of using plain ruby? A minimal test suite is just a collection of test cases - which can be simple methods, or functions, or code blocks, or whatever works.

The most important thing test framework provides is a test runner, which runs each test case, gathers results, and reports them. What could be possible results of a test case?

• Test case could pass
• Test case could have test assertion which fails
• Test case could crash with an error

And here's where everything went wrong. For silly historical reasons test frameworks decided to treat test assertion failure as if it was test crashing with an error. This is just insane.

Here's a tiny toy test, it's quite compact, and reads perfectly fine:

it "Simple names are treated as first/last" do
  user = NameParser.parse("Mike Pence")
  expect(user.first_name).to eq("Mike")
  expect(user.middle_name).to eq(nil)
  expect(user.last_name).to eq("Pence")
end

If assertion failures are treated as failures, and first name assertion fails, then we still have no idea what the code actually returned, and at this point developer will typically run binding.pry or equivalent just to mindlessly copy and paste checks which are already in the spec!

We want the test case to keep going, and then all assertion failures to be reported afterwards!

Common workarounds

There's a long list of workarounds. Some people go as far as recommending "one assertion per test" which is an absolutely awful idea which would result in enormous amounts of boilerplate and hard to read disconnected code. Very few real world projects follow this:

describe "Simple names are treated as first/last" do
  let(:user) { NameParser.parse("Mike Pence") }

  it do
    expect(user.first_name).to eq("Mike")
  end

  it do
    expect(user.middle_name).to eq(nil)
  end

  it do
    expect(user.last_name).to eq("Pence")
  end
end

RSpec has some shortcuts for writing this kind of one assertion tests, but the whole idea is just misguided, and very often it's really difficult to twist test case into a sets of reasonable "one assertion per test" cases, even disregarding code bloat, readability, and performance impact.

Another idea is to collect all tests into one. As vast majority of assertions are simple equality checks, this usually sort of works:

it "Simple names are treated as first/last" do
  user = NameParser.parse("Mike Pence")
  expect([user.first_name, user.middle_name, user.last_name])
    .to eq(["Mike", nil, "Pence])
end




Not exactly amazing code, but at least it's compact.

Actually...

What if test framework was smart enough to keep going after assertion failure? Turns out RSpec can do just that, but you need to explicitly tell it to be sane, by putting this in your spec/spec_helper.rb:

RSpec.configure do |config|
  config.define_derived_metadata do |meta|
    meta[:aggregate_failures] = true
  end
end

And now the code we always wanted to write magically works! If parser fails, we see all failed assertions listed. This really should be on by default.

Limitations

This works with expert and should syntax, and doesn't clash with any commonly used RSpec functionality.

It does not work with config.expect_with :minitest, which is how you can use assert_equal syntax with RSpec test driver. It's not a common thing to do, other than to help migration from minitest to RSpec, and there's no reason why it couldn't be made to work in principle.

What else can it do?

You can write a whole loop like:

it "everything works" do
  collection.each do |example|
    expect(example).to be_valid
  end
end

And if it fails somehow, you'll get a list of failing examples only in test report!

What if I don't like the RSpec syntax?

RSpec syntax is rather controversial, with many fans, but many other people very intensely hating it. It changed multiple times during its existence, including:

user.first_name.should equal("Mike")
user.first_name.should == "Mike"
user.first_name.should eq("Mike")
expect(user.first_name).to eq("Mike")

And in all likelihood it will continue changing. RSpec sort of supports more traditional expectation syntax as a plugin, but it currently doesn't support failure aggregation:

assert_equal "Mike", user.first_name

When I needed to mix them for migration reasons I just defined assert_equal manually, and that was good enough to handle vast majority of tests.

In long term perspective, I'd of course strongly advise every other test frameworks in every language to abandon the historical mistake of treating test assertion failures as errors, and to switch to this kind of failure aggregation.

Considering how much time a typical developer spends dealing with failing tests, even this modest improvement in the process can result in significantly improved productivity.