_ruby

A brief primer on Good Test Writing

In order to write tests that work well in parallel there are some simple guidelines to follow:

Write atomic and autonomous tests
Group like tests together in small batches
Be descriptive
Use a Test Runner

Atomic & Autonomous Tests

Each test needs to be concise (e.g. testing a single feature rather than multiple features) and be able to be run independently (e.g. sets up it's own data rather than relying on a previous test to do it). Doing this may require a mental shift, discipline, and more up front effort. But it will make a dramatic impact on the quality, effectiveness, and maintainability of your tests.

Grouping Tests

As your test suite grows you should have multiple test files, each containing a small grouping of tests broken out by functionality that they're exercising. This will go a long way towards organization and maintenance as your test suite grows -- as well as faster execution times (depending on your approach to parallelization).

Being Descriptive

Each test file should be named appropriately, and each test within it should have an informative name (even if it may be a bit verbose). Also, each test (or grouping of tests) should be tagged for flexible execution later (e.g. on a Continuous Integration server).

This way you can run parts of your test suite as needed, and the results will be informative thanks to helpful naming.

Test Runners

At the heart of every testing framework is a test runner that does a lot of the heavy lifting (e.g. test group execution, easy global configuration for setup and teardown, reporting, etc.).

So rather than reinvent the wheel, you can use one of the many that already exists (there's more than one for every language). And with it, you can bolt on third party libraries to extend it's functionality if there's something missing -- like parallelization.

Example

NOTE: For this example we will be using RSpec as the Test Runner and parallel-tests for parallelization.

In RSpec, test files are referred to as 'specs'. So for our initial spec I've used the test code from a previous tip on working with Data Tables. Don't get too distracted by what the test steps are doing, the organization and naming of things is more important for this example.

# data_table_sorting_spec.rb

require 'selenium-webdriver'

describe 'Data Table Sorting' do

  before(:each) do
    @driver = Selenium::WebDriver.for :firefox
  end

  after(:each) do
    @driver.quit
  end

  context 'Without Attributes' do

    it 'in Ascending Order' do
      @driver.get 'http://the-internet.herokuapp.com/tables'
      @driver.find_element(css: '#table1 thead tr th:nth-of-type(4)').click
      dues = @driver.find_elements(css: '#table1 tbody tr td:nth-of-type(4)')
      due_values = []
      dues.each { |due| due_values << due.text.gsub(/\$/,'').to_i }
      (due_values == due_values.sort).should == true
    end

    it 'in Descending Order' do
      @driver.get 'http://the-internet.herokuapp.com/tables'
      @driver.find_element(css: '#table1 thead tr th:nth-of-type(4)').click
      @driver.find_element(css: '#table1 thead tr th:nth-of-type(4)').click
      dues = @driver.find_elements(css: '#table1 tbody tr td:nth-of-type(4)')
      due_values = []
      dues.each { |due| due_values << due.text.gsub(/\$/,'').to_i }
      (due_values == due_values.sort).should == false
    end

  end

  context 'With Attributes' do

    it 'in Ascending Order' do
      @driver.get 'http://the-internet.herokuapp.com/tables'
      @driver.find_element(css: '#table2 thead .dues').click
      dues = @driver.find_elements(css: '#table2 tbody .dues')
      due_values = []
      dues.each { |due| due_values << due.text.gsub(/\$/,'').to_f }
      (due_values == due_values.sort).should == true
    end

    it 'in Descending Order' do
      @driver.get 'http://the-internet.herokuapp.com/tables'
      @driver.find_element(css: '#table2 thead .dues').click
      @driver.find_element(css: '#table2 thead .dues').click
      dues = @driver.find_elements(css: '#table2 tbody .dues')
      due_values = []
      dues.each { |due| due_values << due.text.gsub(/\$/,'').to_f }
      (due_values == due_values.sort).should == false
    end

  end

end

At the top of the file we include our dependent library (selenium-webdriver) and declare the name of our test group ('Sort Data Table') with a describe statement. We then wire up our before and after actions which are responsible for setting up and tearing down an instance of Selenium for each test. Each of our tests are grouped into logical separations using context blocks and defined within an it block.

If we run this (e.g. with the command rspec data_table_sorting_spec.rb) it will fire up one browser at a time until each test is completed. And If we download, install, and run it through parallel_tests (e.g. with the command paralell_rspec data_table_sorting_spec.rb) -- the same thing would happen. Why? Because parallel_tests executes things at the file level rather than at the individual test level (this holds true even when running tests by tag).

In order to take advantage of parallelization we will need to break these tests up into different files.

But before we do that we can simplify our test code by abstracting out our setup and teardown actions into a central place (known as a spec_helper in the RSpec parlance).

# spec_helper.rb

require 'selenium-webdriver'

RSpec.configure do |config|

  config.before(:each) do
    @driver = Selenium::WebDriver.for :firefox
  end

  config.after(:each) do
    @driver.quit
  end

end

Now that we have that we can create a new spec file and place some of our tests in it.

# data_table_sorting_with_attributes_spec.rb

require 'spec_helper'

describe 'Sort Data Table' do

  context 'With Attributes' do

    it 'in Ascending Order' do
      @driver.get 'http://the-internet.herokuapp.com/tables'
      @driver.find_element(css: '#table2 thead .dues').click
      dues = @driver.find_elements(css: '#table2 tbody .dues')
      due_values = []
      dues.each { |due| due_values << due.text.gsub(/\$/,'').to_f }
      (due_values == due_values.sort).should == true
    end

    it 'in Descending Order' do
      @driver.get 'http://the-internet.herokuapp.com/tables'
      @driver.find_element(css: '#table2 thead .dues').click
      @driver.find_element(css: '#table2 thead .dues').click
      dues = @driver.find_elements(css: '#table2 tbody .dues')
      due_values = []
      dues.each { |due| due_values << due.text.gsub(/\$/,'').to_f }
      (due_values == due_values.sort).should == false
    end

  end

end

Notice that the require statement has changed from selenium-webdriver to spec_helper. Since we abstracted things out into a spec_helper file we now need to require it in all of our specs -- and the dependent libraries for our test suite live there instead.

# data_table_sorting_without_attributes_spec.rb

require 'spec_helper'

describe 'Sort Data Table' do

  context 'Without Attributes' do

    it 'in Ascending Order' do
      @driver.get 'http://the-internet.herokuapp.com/tables'
      @driver.find_element(css: '#table1 thead tr th:nth-of-type(4)').click
      dues = @driver.find_elements(css: '#table1 tbody tr td:nth-of-type(4)')
      due_values = []
      dues.each { |due| due_values << due.text.gsub(/\$/,'').to_i }
      (due_values == due_values.sort).should == true
    end

    it 'in Descending Order' do
      @driver.get 'http://the-internet.herokuapp.com/tables'
      @driver.find_element(css: '#table1 thead tr th:nth-of-type(4)').click
      @driver.find_element(css: '#table1 thead tr th:nth-of-type(4)').click
      dues = @driver.find_elements(css: '#table1 tbody tr td:nth-of-type(4)')
      due_values = []
      dues.each { |due| due_values << due.text.gsub(/\$/,'').to_i }
      (due_values == due_values.sort).should == false
    end

  end

end

To get this last file we could have renamed the original test file, or created it anew and deleted the original.

Notice that the describe name is the same between test files (e.g. 'Sort Data Table'). In RSpec names do not need to be unique, and these tests logically fit together, this is a welcome consistency.

Now when we run our tests through parallel_tests we get concurrent test runs with two browsers firing at the same time, yielding (in this case) roughly a 30% drop in execution times (e.g. 17 seconds down to 12 seconds). You can easily up the number of processes being run as a command-line argument (e.g. -n 5). But this will only make a difference as you have more specs given how parallel_tests groups and executes things.

Now that we have a working example that runs concurrently, we can point it at a cloud offering like Sauce Labs, and have access to any browser we want. We just have to update our setup action in the spec_helper.

NOTE: This was covered in a previous tip. For a full write-up, go here.

# spec_helper.rb

require 'selenium-webdriver'

RSpec.configure do |config|

  config.before(:each) do
    caps = Selenium::WebDriver::Remote::Capabilities.firefox
    caps.browser_version = "115"
    caps.platform_name = "Windows 11"
    caps[:name] = self.example.metadata[:full_description]

    @driver = Selenium::WebDriver.for(
      :remote,
      :url => "https://SAUCE_USERNAME:SAUCE_API_KEY@ondemand.saucelabs.com/wd/hub",
      :capabilities => caps)
  end

  config.after(:each) do
    @driver.quit
  end

end

Now when we run our tests with parallel_tests we can see the concurrent execution happening in Sauce Labs.

Sauce Labs is built to handle your tests concurrently, so you should be able to turn the number of processes up with no sweat. You will just need to be cogniscent of how many parallel tests your account has access to (e.g. 3 for Sauce Open accounts).

Expected Behavior

Several browsers open in parallel
Each test runs in a different browser
All browsers close
If Sauce Labs was used, then the test results (along with a video recording, screenshots, and other debugging information) are available on the test results dashboard.

Summary

Hopefully this tip has helped steer you on a path towards better test writing and parallelization.

Stay tuned for future tips where we'll cover how to take full advantage of your test suite by wiring it into a Continuous Integration server and making the requisite changes to your test runner.

Happy Testing!

A brief primer on Good Test Writing​

Atomic & Autonomous Tests​

Grouping Tests​

Being Descriptive​

Test Runners​

Example​

Expected Behavior​

Summary​