_python

Example

Our example page is from the-internet and can be seen here. There are three different versions of the page that are available. On each page the heading text will vary:

• Control: A/B Test Control
• Variation 1: A/B Test Variation 1
• Opt-out: No A/B Test

Let's start things off by loading our requisite libraries (import unittest for our test framework and from selenium import webdriver to drive the browser), declare our test class, and wire up some test setUp and tearDown methods.

# filename: ab_test_opt_out.py
import unittest
from selenium import webdriver
from selenium.webdriver.common.by import By


class ABTestOptOut(unittest.TestCase):

    def setUp(self):
        self.driver = webdriver.Firefox()

    def tearDown(self):
        self.driver.quit()
# ...

Now let's begin our first test to step through loading the split testing page and verifying that the text changes after we forge an opt-out cookie.

# filename: ab_test_opt_out.py
# ...
    def test_forge_cookie_on_target_page(self):
        driver = self.driver
        driver.get('http://the-internet.herokuapp.com/abtest')
        heading_text = driver.find_element(By.TAG_NAME,'h3').text
        assert heading_text in ['A/B Test Variation 1', 'A/B Test Control']
        driver.add_cookie({'name' : 'optimizelyOptOut', 'value' : 'true'})
        driver.refresh()
        heading_text = driver.find_element(By.TAG_NAME,'h3').text
        assert heading_text == 'No A/B Test'

# ...

After navigating to the page we confirm that we are in one of the A/B test groups by grabbing the heading text and checking to see if it matches what we expect. After that we add the opt-out cookie, refresh the page, and then confirm that we are no longer in the A/B test group by checking the heading text again.

We could also load the opt-out cookie before navigating to this page.

# filename: ab_test_opt_out.py
# ...
    def test_forge_cookie_on_homepage_then_navigate_to_target_page(self):
        driver = self.driver
        driver.get('http://the-internet.herokuapp.com')
        driver.add_cookie({'name' : 'optimizelyOptOut', 'value' : 'true'})
        driver.get('http://the-internet.herokuapp.com/abtest')
        heading_text = driver.find_element(By.TAG_NAME,'h3').text
        assert heading_text == 'No A/B Test'

# ...

Here we are navigating to the main page of the site first and then adding the opt-out cookie. After that we navigate to the split test page and then perform our check. Alternatively, we could opt out without forging a cookie. Instead, we just need to append an opt out parameter to the URL.

# filename: ab_test_opt_out.py
    def test_url_parameter(self):
            driver = self.driver
            driver.get('http://the-internet.herokuapp.com/abtest?optimizely_opt_out=true')
            driver.switch_to.alert.dismiss()
            heading_text = driver.find_element(By.TAG_NAME,'h3').text
            assert heading_text == 'No A/B Test'

if __name__ == "__main__":
    unittest.main()

By appending ?optimizely_opt_out=true we achieve the same outcome as before. Keep in mind that this approach triggers a JavaScript alert, so we have to switch to and dismiss it (e.g., driver.switch_to.alert.dismiss()) before performing our check.

Expected Behavior

When we save this file and run it (e.g., python ab_test_opt_out.py from the command-line) here is what will happen with either of the tests:

• Open the browser
• Opt-out of the split tests (either by cookie or appended URL)
• Visit the split testing page
• Grab the header text
• Confirm that the session is opted out of the split test
• Close the browser

Summary

Hopefully this tip was helpful in guiding you in different methods of how to opt out of split tests.

Happy Testing!