How To Retrieve HTTP Status Codes

The Problem

There are times when you are testing a page (or numerous pages) and you want to verify that it responded correctly. A great way to handle this is by checking the HTTP Status Code that the browser received. However this functionality is not available in Selenium WebDriver.

A Solution

The tried and true approach that Selenium Committers and Practitioners recommend is to use a proxy server. With it, you will be able to watch and manipulate network traffic to and from the application you're testing. Thus giving you access to a whole host of functionality that isn't available otherwise.

Let's step through an example using BrowserMob Proxy.

An Example

First we need to download a copy of BrowserMob Proxy.

Next we'll create a script and pull in our requisite libraries (e.g., selenium-webdriver to drive the browser, rspec/expectations and it's matchers for our assertion, and the browsermob/proxy library to control the proxy server).

# filename: status_codes.rb

require 'selenium-webdriver'
require 'rspec/expectations'
include RSpec::Matchers
require 'browsermob/proxy'

Now let's create a method to prepare the proxy for use with Selenium. In it we start the proxy server, configure a browser profile to use it (for Firefox in this case), and return the profile object.

def configure_proxy
  server = BrowserMob::Proxy::Server.new(
    File.join(Dir.pwd, 'browsermob-proxy-2.0.0/bin/browsermob-proxy'))
  @proxy = server.start.create_proxy
  profile = Selenium::WebDriver::Firefox::Profile.new
  profile.proxy = @proxy.selenium_proxy

Next we wire up some simple setup, teardown, and run methods to handle our test configuration.

def setup
  @driver = Selenium::WebDriver.for :firefox, profile: configure_proxy

def teardown

def run

In setup we use the configure_proxy method when specifying the profile object. And we close the proxy in teardown (after quitting the Selenium session).

Now let's create a helper method to pull the status code out of the browser's HTTP Archive (a.k.a. HAR) when an action is performed on the page.

def retrieve_status_code

Now we're ready to wire up our test.

run do
  status_code = retrieve_status_code do
    @driver.get 'http://the-internet.herokuapp.com/status_codes/404'

  expect(status_code).to eql 404

When visiting the page we are passing the Selenium command to the retrieve_status_code method which returns the HTTP response code. We store this in a status_code variable and use it to check that the response code is what we expect (404).

Expected Behavior

If you save this file and run it (e.g., ruby status_codes.rb from the command-line) here is what will happen:

  • Proxy server starts
  • Proxy server session created
  • Browser opens
  • Visit the URL
  • Retrieve the HTTP Status Code from loading the page
  • Check that the status code is what was expected
  • Browser closes
  • Proxy session closes


If you get an error when running your test, be sure to append log: true when creating an instance of Browsermob::Proxy::Server.

  server = BrowserMob::Proxy::Server.new(
    File.join(Dir.pwd, 'browsermob-proxy-2.0.0/bin/browsermob-proxy'), log: true)

Now when you run your test, you will see more detailed information as to why the server was unable to start.


This tip was inspired by Jim Evans' multi-part blog post series on doing the same thing in C# with Fiddler (1, 2, 3) which was in response to Selenium Issue 141. Thanks Jim!

Happy Testing!

Back to the archives