RSpec 3.4 has been released!

Myron Marston

Nov 13, 2015

RSpec 3.4 has just been released! Given our commitment to semantic versioning, this should be a trivial upgrade for anyone already using any RSpec 3 release, but if we did introduce any regressions, please let us know, and we’ll get a patch release out with a fix ASAP.

RSpec continues to be a community-driven project with contributors from all over the world. This release includes over 500 commits and 160 merged pull requests from nearly 50 different contributors!

Thank you to everyone who helped make this release happen!

Notable Changes

Core: Bisect Algorithm Improvements

RSpec 3.3 shipped with a new --bisect option that identifies a minimal reproduction command when you are tracking down the source of an ordering dependency. The core bisection algorithm used a naive permutations approach: in each round, first it would try one half of the examples, than the other half, and then each combination of half of the examples, until it found a half it could safely ignore. This generally worked OK, but had some horrible worst case behavior. In particular, if your multiple culprits were involved in the ordering dependency, it could take many combinations to hit on one that happened to contain both. Likewise, if the algorithm had hit the point where more than half of the remaining examples were culprits, it would exhaustively try every combination until none remained – which would take a very long time.

In RSpec 3.4, the bisection algorithm is much more intelligent. It now uses a recursive approach that is designed to minimize the number of attempts needed to identify the culprits. Early feedback on the new algorithm is quite positive: Sam Livingston-Gray reported that a 3.3 bisection ran all night without completing, but with the new algorithm it completed in only 20 minutes!

Thanks to Simon Coffey for implementing this! If you’d like to learn more, I recommend you check out his PR – it contains some really useful diagrams explaining how the new algorithm works.

Core: Failure Output Improvements

Good failure output has always been a priority for RSpec, but in 3.4 it’s much improved, in a few ways:

Multi-line Code Snippets

RSpec includes a code snippet from the expectation failure in the failure output. Before RSpec 3.4, this generally worked OK as long as your expectation fit on one line. If you formatted it across multiple lines, like this:

expect {
  MyNamespace::MyClass.some_long_method_name(:with, :some, :arguments)
}.to raise_error(/some error snippet/)

…then the failure would just print the first line (expect {) since that’s the stackframe included in the exception. In RSpec 3.4, we now load Ripper from the standard library if it’s available in order to parse the source and determine how many lines to include for the full expectation expression. For a case like the snippet above, the failure output will now include the entire expression.

There’s also a new config option to go wth this: config.max_displayed_failure_line_count, which defaults to 10 and sets a limit on the size of the snippet.

Thanks to Yuji Nakayama for implementing this!

Install coderay for Syntax Highlighting

Taking this a step further, if the coderay gem is available, RSpec 3.4 will use it to syntax highlight the code snippet in your terminal. Here’s an example of how that looks, using the code snippet from above:

Failure with syntax highlighting

Better Failure Source Detection

RSpec finds the failure code snippet by looking through the exception stack trace for an appropriate frame. We could just use the top stack frame but that’s generally not what you want: when you have an expectation failure, the top frame refers to a link in RSpec where the RSpec::Expectations::ExpectationNotMetError was raised, and you want to see the snippet from your expect call site rather than seeing a snippet of RSpec’s code. Before RSpec 3.4, our solution for this was fairly naive: we just looked for the first stack frame from the spec containing your current running example file. In some situations this would display the wrong snippet (such as when your example called a helper method defined in a spec/support file where the real failure occurred). In others it didn’t find anything and we wound up displaying Unable to find matching line from backtrace instead.

RSpec 3.4 has much better logic for finding the source snippet: now we look for the first frame from the config.project_source_dirs (defaults to lib, app and spec) and if no matching frame can be found, we fall back to the first stack frame. You shouldn’t see Unable to find matching line from backtrace anymore!

Expectations: Better Compound Failure Messages

Continuing with the “improved failure output” theme, rspec-expectations 3.4 provides better failure messages for compound expectations. Before, we would combine each failure message into a single line. For example, this expectation:

expect(lyrics).to start_with("There must be some kind of way out of here")
              .and include("No reason to get excited")

…produced this hard-to-read failure:

1) All Along the Watchtower has the expected lyrics
   Failure/Error: expect(lyrics).to start_with("There must be some kind of way out of here")
     expected "I stand up next to a mountain And I chop it down with the edge of my hand" to start with "There must be some kind of way out of here" and expected "I stand up next to a mountain And I chop it down with the edge of my hand" to include "No reason to get excited"
   # ./spec/example_spec.rb:20:in `block (2 levels) in <top (required)>'

In RSpec 3.4, we format separate each individual failure message so it’s easier to read:

1) All Along the Watchtower has the expected lyrics
   Failure/Error:
     expect(lyrics).to start_with("There must be some kind of way out of here")
                   .and include("No reason to get excited")

        expected "I stand up next to a mountain And I chop it down with the edge of my hand" to start with "There must be some kind of way out of here"

     ...and:

        expected "I stand up next to a mountain And I chop it down with the edge of my hand" to include "No reason to get excited"
   # ./spec/example_spec.rb:20:in `block (2 levels) in <top (required)>'

Expectations: Add with_captures to match matcher

In RSpec 3.4, the match matcher has gained a new ability: you can specify regex captures. You can use the new with_captures method to specify ordered captures:

year_regex = /(\d{4})\-(\d{2})\-(\d{2})/
expect(year_regex).to match("2015-12-25").with_captures("2015", "12", "25")

…or to specify named captures:

year_regex = /(?<year>\d{4})\-(?<month>\d{2})\-(?<day>\d{2})/
expect(year_regex).to match("2015-12-25").with_captures(
  year: "2015",
  month: "12",
  day: "25"
)

Thanks to Sam Phippen and Jason Karns who collaborated on this new feature.

Rails: New have_enqueued_job matcher for ActiveJob

Rails 4.2 shipped with ActiveJob, and rspec-rails 3.4 now has a matcher that allows you to specify that a block of code enqueues a job. It supports a fluent interface that will look familiar if you’re an rspec-mocks user:

expect {
  HeavyLiftingJob.perform_later
}.to have_enqueued_job

expect {
  HelloJob.perform_later
  HeavyLiftingJob.perform_later
}.to have_enqueued_job(HelloJob).exactly(:once)

expect {
  HelloJob.perform_later
  HelloJob.perform_later
  HelloJob.perform_later
}.to have_enqueued_job(HelloJob).at_least(2).times

expect {
  HelloJob.perform_later
}.to have_enqueued_job(HelloJob).at_most(:twice)

expect {
  HelloJob.perform_later
  HeavyLiftingJob.perform_later
}.to have_enqueued_job(HelloJob).and have_enqueued_job(HeavyLiftingJob)

expect {
  HelloJob.set(wait_until: Date.tomorrow.noon, queue: "low").perform_later(42)
}.to have_enqueued_job.with(42).on_queue("low").at(Date.tomorrow.noon)

Thanks to Wojciech Wnętrzak for implementing this feature!

Stats

Combined:

rspec-core:

rspec-expectations:

rspec-mocks:

rspec-rails:

rspec-support:

Docs

API Docs

Cucumber Features

Release Notes

rspec-core 3.4.0

Full Changelog

Enhancements:

Bug Fixes:

rspec-expectations 3.4.0

Full Changelog

Enhancements:

Bug Fixes:

rspec-mocks 3.4.0

Full Changelog

Enhancements:

Bug Fixes:

rspec-rails 3.4.0

Full Changelog

Enhancements:

Bug Fixes:

rspec-support 3.4.0

Full Changelog

Enhancements:

Bug Fixes: