The whole purpose of writing test suites is to ensure that code is running as expected. But, we can only really know that it’s running as expected if our test suites actually test all of our code. Enter code coverage! Most languages have libraries to support telling us how much of our code was actually exercised by our tests, which can help identify places where the code is never run as part of tests. It’s not necessarily always the goal to get 100% code coverage, but we can use these libraries to make informed decisions about where our testing might be lacking, and then decide if we should have tests for that code.

Ruby’s standard library has a Coverage Module that we can dig into. For our example, let’s say we’re trying to solve the problem of changing the case of a word. If it’s uppercase, we want our method to return it lowercase, and if it’s lowercase, we’ll return it uppercase.

We can use Ruby’s ternary if to write this in one line. The syntax is: <condition> ? <true_case> : <false_case>

class CaseConverter
  def self.switch_case(word)
    word == word.upcase ? word.downcase : word.upcase
  end
end

Let’s also write a small test to accompany it which only covers one case (word → WORD). This will be a good check for our coverage, because we expect that we won’t have full coverage:

RSpec.describe CaseConverter do
  describe ".switch_case" do
    subject { described_class.switch_case(word) }
    context "when word is lowercase" do
      let(:word) { "word" }
      it "becomes uppercase" do
        expect(subject).to eq("WORD")
      end
    end
  end
end

We’ll also need a snippet to run the spec and tell us the coverage of our test file. The Ruby docs for coverage say “A coverage array gives, for each line, the number of line executions by the interpreter.” So we’ll assert there’re no 0s in that array:

require "coverage"
require "rspec"

Coverage.start
RSpec::Core::Runner.run(["case_converter_spec.rb"])

all_lines_covered = !Coverage.result["case_converter.rb"].any?(0)
puts "All lines are covered: #{all_lines_covered}"

and we can run it against our test file and see:

.
1 example, 0 failures

All lines are covered: true

But… wait… we only wrote a test for converting a lowercase word into an uppercase word. And we’re seeing full coverage?! Plenty of developers operate with too much false confidence already, the last thing we need is a coverage module incorrectly supporting that.

This actually our ternary if coming back to haunt us! The tests are technically covering every line of our code. It’s just that we have two different cases within one line. (We’d see the same thing using modifier ifs.)

The documentation for Coverage.start is not incredibly thorough about all of the arguments we can pass through, but if we look at the source code we see that we can pass branches: true as an argument to see branch coverage. Neat. This sounds like exactly what we want, and should cover our ternary if to give us a full picture of our coverage. Let’s refactor our coverage runner slightly to instead look at branch coverage:

Coverage.start(branches: true)
RSpec::Core::Runner.run(["case_converter_spec.rb"])

all_branches_covered =
  !Coverage.result["case_converter.rb"][:branches].values.flat_map(&:values).any?(0)
puts "All branches are covered: #{all_branches_covered}"

and run it:

.
1 example, 0 failures

All branches are covered: false

So our hypothesis about the ternary if is correct, we can see that we don’t have full branch coverage. When looking at line coverage, we were under the impression that we had full coverage, when really there was one branch we weren’t testing!

Let’s add that test case (WORD → word):

context "when a word is uppercase" do
  let(:word) { "WORD" }
  it "becomes lowercase" do
    expect(subject).to eq("word")
  end
end

and confirm we now have full coverage:

...
2 examples, 0 failures

All branches are covered: true

Great! But, there are still two things we need to address:

  1. Our method is slightly misleading. If a word is fully uppercase, it will return the lowercase word, but if it has a mix of lowercase and uppercase letters, our method will return it all in uppercase
  2. There must be a reason we still need line coverage, just branch alone can’t be sufficient or that would be the default

Let’s tackle the first one, and instead change case based on whichever case is the majority. If the majority of characters are uppercase, we’ll return the word lowercase, and vice versa. In a tie, we’ll arbitrarily return the word in uppercase. (In this refactor, we’ll also sneakily answer #2.)

class CaseConverter
  def self.switch_case_by_majority(word)
    uppercase_count = 0

    word.each_char do |char|
      uppercase_count += 1 if char.upcase == char

      # Small optimization to return as soon as we know that majority of
      # characters are uppercase. In the case of an extremely long word, this
      # would mean we wouldn't have to iterate over all characters
      return word.downcase if uppercase_count > word.length / 2
    end

    # If we get here, our word was not majority uppercase at
    # any point (above), so we know it's mostly lowercase and should flip it
    word.upcase
  end
end

and, just like we did last time, let’s write a little test (wORD → word) which does not cover the full spectrum of test cases, so we can confirm we’re not seeing full coverage:

RSpec.describe CaseConverter do
  describe ".switch_case_by_majority" do
    subject { described_class.switch_case_by_majority(word) }

    context "when word has majority uppercase letters" do
      let(:word) { "wORD" }

      it "becomes lowercase" do
        expect(subject).to eq("word")
      end
    end
  end
end

running our coverage checker now with branches we get:

.
1 examples, 0 failures

All branches are covered: true

Full branch coverage?! Looking back at our method, we are actually covering all branch cases. But we’re never executing the last word.upcase because of our optimization to return early. These comments explain how:

class CaseConverter
  def self.switch_case_by_majority(word)
    uppercase_count = 0

    word.each_char do |char|
      # Executes both branch cases on input `wORD`:
      #   conditional is false for `w` (`W` != `w`)
      #   conditional is true for `O` (`O` == `O`)
      uppercase_count += 1 if char.upcase == char

      # Executes both branch cases on input `wORD`:
      #
      #   conditional is false when we've iterated over just
      #   `w` because uppercase_count will be 0 and 0 < 2
      #
      #   conditional is true when we've iterated over each
      #   character in `wORD` because uppercase_count will be 3
      #   and 3 > 2
      return word.downcase if uppercase_count > word.length / 2
    end

    # We are NEVER getting here with input `wORD`
    word.upcase
  end
end

What we want here is line coverage. Running that again, we see:

.
1 examples, 0 failures

All lines are covered: false

Aha! Let’s now write a test for the when a word is majority lowercase (wOrd → WORD):

context "when word has fewer than majority uppercase letters" do
  let(:word) { "wOrd" }

  it "becomes uppercase" do
    expect(subject).to eq("WORD")
  end
end

and running that, we’ll now see:

..
2 examples, 0 failures

All lines are covered: true

Interesting. For the first method, line coverage was not telling us the full story, due to our ternary operator, but branch coverage was. And for the second method, branch coverage was not telling us the full story, due to an optimized early return, but line coverage was.

Our conclusion then is that in order to properly make assertions about our coverage in Ruby, we need to measure both branch coverage and line coverage. These together will give us the full picture of whether every line of code is executed, and within those lines, whether every possible case was executed.

If you’re interested in reading more about branch coverage in Ruby, check out the README for the SimpleCov gem.