Skip to content

Selenium WebDriver + HP ALM Integration Guide

Selenium WebDriver is one of the most widely used open source tool for automating browser interaction and is used by countless companies.

Bumblebee's JUnit and TestNG solution allows you to easily integrate Selenium WebDriver tests with HP ALM, map them to requirements, and automatically upload tests, and test results to HP ALM Testplan, TestLab, and Requirements Module.

Install Bumblebee Server before configuring maven projects

Bumblebee's JUnit and TestNG solution communicates with HP ALM via the Bumblebee server. You must install Bumblebee server before configuring your maven project. Bumblebee server setup instructions

Project Setup

Maven

Modify your pom.xml file and add agiletestware repository and bumblebee-annotations dependency:

<dependencies>
    <dependency>
        <groupId>com.agiletestware</groupId>
        <artifactId>bumblebee-annotations</artifactId>
        <version>0.0.9</version>
        <scope>test</scope>
    </dependency>
</dependencies>
<repositories>
    <repository>
        <id>release.agiletestware.com</id>
        <url>http://public.maven.agiletestware.com.s3-website-us-west-2.amazonaws.com/release</url>
    </repository>
</repositories>
Configure Maven surefire plugin for JUnit or TestNG frameworks.

JUnit Maven surefire plugin config: 

<build>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>2.19</version>
            <configuration>
                <properties>
                    <property>
                        <name>listener</name>
                        <value>com.agiletestware.bumblebee.annotations.BumblebeeJUnitListener</value>
                    </property>
                </properties>
            </configuration>
        </plugin>
    </plugins>
</build>

TestNG Maven surefire plugin config: 

<build>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>2.19</version>
            <configuration>
                <properties>
                    <property>
                        <name>listener</name>
                        <value>com.agiletestware.bumblebee.annotations.testng.BumblebeeTestNGReporter</value>
                    </property>
                </properties>
            </configuration>
        </plugin>
    </plugins>
</build>

Configuration file

Bumblebee configuration file contains information about your HP ALM setup. The configuration values are read by Bumblebee's JUnit and TestNG solution and used to update test results in HP ALM.

Create bumblebee_config.xml file and put into the project root folder. Example:

<?xml version="1.0"?>
<bumblebee>
    <!-- URL of the Bumblebee Server -->
    <bumblebee_url>http://server_name:port/bumblebee</bumblebee_url>
    <!-- URL of HP ALM Server -->
    <alm_url>http://server_name:port/qcbin</alm_url>
    <!-- Name of HP ALM User -->
    <alm_user>qcuser</alm_user>
    <!-- Encrypted password: please use http://server_name:port/bumblebee/password/encrypt to encrypt your plain text password  -->
    <alm_encrypted_pass>fd4OMOXLJjkMR6e64RJh3Q==</alm_encrypted_pass>
    <!-- HP ALM Domain -->
    <alm_domain>DEFAULT</alm_domain>
    <!-- HP ALM Project -->
    <alm_project>annotations_demo</alm_project>
    <!-- Asynchronous (offline) update -->
    <async_update>true</async_update>
</bumblebee>

Bumblebee Annotations

Bumblebee provides specialized annotations package that allows users to map JAVA tests to ALM tests and export their test results. @Bumblebee Java annotation can be added on JUnit/TestNG class or method level.

Annotation parameters

Name Description Applicable Required
testplan Path to test in a Test Plan in HP ALM. e.g. Subject\Test1 Class, method Yes
testlab Path to test set in a test lab in HP ALM. e.g. Root\Test1 Class, method Yes
testset Name of test set in HP ALM Class, method Yes
testname If specified, then the value of testname parameter will be set as the,name of test in HP ALM. If not set, test name will be set to fully,qualified method name, e.g. com.annotations.Demo.method1 Method No
almid Defines id of a test in a HP ALM test plan which needs to be updated. If specified then testplan and testname are ignored Method No
description Description for test in HP ALM Method No
overwriteAlmSteps If set to BooleanValue.TRUE, all existing test steps will be deleted Class, method No
parameters An array of custom parameters which are passed to the bumblebee server and then mapped to HP ALM custom fields. Please refer to documentation documentation on how to setup mappings on Bumblebee server Class, method No
hasSteps Works with TestNG only If set to BooleanValue.TRUE, test class is exported as a single test in HP ALM, and all its test methods are exported as test steps within the test No

Bumblebee annotation on method level overrides values set by Bumblebee annotation on class level.

To run JUnit/TestNG tests and send results to HP ALM, simply run Maven test phase on your project: Example: mvn test

Note

Unless stated, all the examples below are for JUnit. To run the same example on TestNG, simply replace org.junit.Test with org.testng.annotations.Test Imports are omitted for brevity.

Class Level Annotations

In this example, we have added the annotations to the class. This means that all the tests within this class will inherit the same testplan, testlab, testset, etc values.

package com.agiletestware.bumblebee.dummytest;

@Bumblebee(testplan = "Subject\\webdriver", testlab = "Root\\webdriver", testset = "class annotations")
public class WebDriverTest {

    private WebDriver driver;

    @BeforeClass
    public static void beforeClass() {
        System.setProperty("webdriver.chrome.driver", new File("chromedriver.exe").getAbsolutePath());
    }

    @Before
    public void setUp() {
        this.driver = new ChromeDriver();
    }

    @Test
    public void testOne() {
        driver.get("http:\\agiletestware.com");
        Assert.assertEquals("Agiletestware - Software for QA and Development Tools", driver.getTitle());
    }

    @Test
    public void testTwo() {
        driver.get("http:\\agiletestware.com");
        Assert.assertEquals("Something wrong", driver.getTitle());
    }

    @Ignore@Test
    public void ignoredTest() {
        driver.get("http:\\agiletestware.com");
    }

    @Test
    public void testThree() {
        throw new RuntimeException("You shall not pass!!!");
    }

    @After
    public void tearDown() {
        driver.quit();
    }
}

When the above example is executed, the results will be automaticaly posted to HP ALM.

Results in HP ALM

Method Level Annotations

In some situations, you may not want to export all the tests or you may want to group tests in different testplan, testlab, testsets. By applying Bumblebee annotations to methods, the user can have more granular control.

public class WebDriverTest {

    private WebDriver driver;

    @BeforeClass
    public static void beforeClass() {
        System.setProperty("webdriver.chrome.driver", new File("chromedriver.exe").getAbsolutePath());
    }

    @Before
    public void setUp() {
        this.driver = new ChromeDriver();
    }

    @Bumblebee(testname = "testOne", testplan = "Subject\\webdriver", testlab = "Root\\webdriver", testset = "method annotations")@Test
    public void testOne() {
        driver.get("http:\\agiletestware.com");
        Assert.assertEquals("Agiletestware - Software for QA and Development Tools", driver.getTitle());
    }

    @Test
    public void testTwo() {
        driver.get("http:\\agiletestware.com");
        Assert.assertEquals("Something wrong", driver.getTitle());
    }

    @Ignore@Test
    public void ignoredTest() {
        driver.get("http:\\agiletestware.com");
    }

    @Test
    public void testThree() {
        throw new RuntimeException("You shall not pass!!!");
    }

    @After
    public void tearDown() {
        driver.quit();
    }
}

When the above example is executed, only testOne() will be exported to HP ALM.

Results in HP ALM

Overriding annotation values

In some situations, you may class-level annotations but may not want to apply them to all the methods. In such a situation, you can provide both class-level and method-level annotations to override the class-level values.

//class-level annotations values
@Bumblebee(testplan = "Subject\\webdriver", testlab = "Root\\webdriver", testset = "overridden values")
public class WebDriverTest {

    private WebDriver driver;

    @BeforeClass
    public static void beforeClass() {
        System.setProperty("webdriver.chrome.driver", new File("chromedriver.exe").getAbsolutePath());
    }

    @Before
    public void setUp() {
        this.driver = new ChromeDriver();
    }

    @Test
    public void testOne() {
        driver.get("http:\\agiletestware.com");
        Assert.assertEquals("Agiletestware - Software for QA and Development Tools", driver.getTitle());
    }

    //method-level annotations values
    @Bumblebee(testname = "test two", testplan = "Subject\\webdriver", testlab = "Root\\webdriver", testset = "overridden values")@Test
    public void testTwo() {
        driver.get("http:\\agiletestware.com");
        Assert.assertEquals("Something wrong", driver.getTitle());
    }

    @Ignore@Test
    public void ignoredTest() {
        driver.get("http:\\agiletestware.com");
    }

    @Test
    public void testThree() {
        throw new RuntimeException("You shall not pass!!!");
    }

    @After
    public void tearDown() {
        driver.quit();
    }

}

When the above example is executed, testOne() and testThree() will use class-level annotation values. testTwo() will use method-level annotation values.

Results in HP ALM

Map test method to existing test in HP ALM

In many companies, it is very common for business analysts or manual test engineers write manual tests in HP ALM. These manual tests are then automated at a later stage. Bumblebee annotation can be used to help implement this workflow. As tests are automated in JAVA, they can be mapped to the manual tests in HP ALM.

For Example: Here is the existing test in HP ALM Test Plan with id = 112

Map to existing test

It contains three design steps:

Map to existing test

To map this manual ALM test to a test method in JAVA, use the Bumblebee annotation's almid parameter along with the testlab and testset values.

@Bumblebee(testplan = "Subject\\webdriver", testlab = "Root\\webdriver", testset = "overridden values")
public class WebDriverTest {

    private WebDriver driver;

    @BeforeClass
    public static void beforeClass() {
        System.setProperty("webdriver.chrome.driver", new File("chromedriver.exe").getAbsolutePath());
    }

    @Before
    public void setUp() {
        this.driver = new ChromeDriver();
    }

    @Test
    public void testOne() {
        driver.get("http:\\agiletestware.com");
        Assert.assertEquals("Agiletestware - Software for QA and Development Tools", driver.getTitle());
    }

    @Bumblebee(almid = 112, testlab = "Root\\webdriver", testset = "overridden values", description = "description")@Test
    public void testTwo() {
        driver.get("http:\\agiletestware.com");
        Assert.assertEquals("Something wrong", driver.getTitle());
    }

    @Ignore@Test
    public void ignoredTest() {
        driver.get("http:\\agiletestware.com");
    }

    @Test
    public void testThree() {
        throw new RuntimeException("You shall not pass!!!");
    }

    @After
    public void tearDown() {
        driver.quit();
    }

}

In the example above, testTwo() has been mapped to the manual testcase in ALM and also updated the test description using the description annotation parameter.

Result in HP ALM

There is also a new test step with name Test Results:

Result in HP ALM

And results are exported to the HP ALM Test Lab:

Result in HP ALM

Custom Fields Configuration

Most HP ALM projects have some custom user-defined fields. If these fields are required, they must be specified by the user. Fortunately, Bumblebee annotation frameworks make it super easy to configure custom user fields. Bumblebee annotation framework allows users to pass custom parameters to Bumblebee Server which contains the mapping to the custom user fields in HP ALM.

Bumblebee server mapping format

This is the format for specifying ALM's custom user fields in the Bumblebee server configuration. In this example, TS_USER_01 is the actual name of a custom field in ALM, label is the user friendly key that will be used within the JAVA annotations for mapping. 

<mapping alm_field="TS_USER_01" label="user field"/>

Please refer to ALM custom fields for detailed steps on how to setup mappings on Bumblebee server.

Example: Lets say we have a custom text field: TS_USER_01 field in HP ALM and we want to set this field to value from Java. Since we have already mapped this on Bumblebee server, we can use the label value user field in our JAVA annotation to complete the mapping. 

@Bumblebee(testplan = "Subject\\webdriver", testlab = "Root\\webdriver", testset = "overridden values")
public class WebDriverTest {

    private WebDriver driver;

    @BeforeClass
    public static void beforeClass() {
        System.setProperty("webdriver.chrome.driver", new File("chromedriver.exe").getAbsolutePath());
    }

    @Before
    public void setUp() {
        this.driver = new ChromeDriver();
    }

    @Bumblebee(almid = 112, testlab = "Root\\webdriver", testset = "overridden values", description = "description", parameters = {@Parameter(name = "user field", value = "value from Java")
    })@Test
    public void testTwo() {
        driver.get("http:\\agiletestware.com");
        Assert.assertEquals("Something wrong", driver.getTitle());
    }

    @After
    public void tearDown() {
        driver.quit();
    }
}

Once this test is executed, the mappings values are passed from the tests to the Bumblebee server and configured at run time.

Tip

Bumblebee annotations are very flexible and powerful and can be used to set custom fields for HP ALM Test, Test Set and Run.

Exporting all test methods into one HP ALM test (TestNG only)

Sometimes tests classes are designed in such a way that each of them represents a single test case and test methods are just test steps. If you have such a test design, Bumblebee can help map this to HP ALM.

To export all methods of a single test class to HP ALM, mark the class with @Bumblebee annotation with hasSteps attribute set to BooleanValue.TRUE.

Example:

@Bumblebee(testplan = "Subject\\webdriver", testlab = "Root\\webdriver", testset = "overridden values", hasSteps = BooleanValue.TRUE)
public class WebDriverTest {

    private WebDriver driver;

    @BeforeClass
    public static void beforeClass() {
        System.setProperty("webdriver.chrome.driver", new File("chromedriver.exe").getAbsolutePath());
    }

    @BeforeTest
    public void setUp() {
        this.driver = new ChromeDriver();
    }

    @Test(priority = 1)
    public void testOne() {
        driver.get("http:\\agiletestware.com");
        Assert.assertEquals("Agiletestware - Software for QA and Development Tools", driver.getTitle());
    }

    @Test(priority = 2)
    public void testTwo() {
        driver.get("http:\\agiletestware.com");
        Assert.assertEquals("Something wrong", driver.getTitle());
    }

    @Test(enabled = false)
    public void ignoredTest() {
        driver.get("http:\\agiletestware.com");
    }

    @Test(priority = 3)
    public void testThree() {
        throw new RuntimeException("You shall not pass!!!");
    }

    @AfterTest
    public void tearDown() {
        driver.quit();
    }
}

In such a case, the new test with three steps will be created in HP ALM Test Plan:

Result in HP ALM

and results will also appear in Test Lab:

Result in HP ALM

Note

Bumblebee relies on test method execution order provided by TestNG, so you have to provide a fixed test methods execution order somehow (e.g. via priorities). Otherwise your test steps will have incorrect execution order and consequently wrong order in HP ALM.

By default, test step names are the same as names of corresponding test methods in your Java code. To change test step names and some other export settings, you can use the @BumblebeeTestStep. This annotation can be applied to test methods only and has the following properties:

Name Description Required
name Name of a test step in HP ALM Yes
description Description of a test step in HP ALM No
expected Value for Expected field of a test step in HP ALM No

Example:

@Bumblebee(testplan = "Subject\\webdriver", testlab = "Root\\webdriver", testset = "overridden values", hasSteps = BooleanValue.TRUE)
public class WebDriverTest {

    private WebDriver driver;

    @BeforeClass
    public static void beforeClass() {
        System.setProperty("webdriver.chrome.driver", new File("chromedriver.exe").getAbsolutePath());
    }

    @BeforeTest
    public void setUp() {
        this.driver = new ChromeDriver();
    }

    @Test(priority = 1)
    public void testOne() {
        driver.get("http:\\agiletestware.com");
        Assert.assertEquals("Agiletestware - Software for QA and Development Tools", driver.getTitle());
    }

    @Test(priority = 2)
    public void testTwo() {
        driver.get("http:\\agiletestware.com");
        Assert.assertEquals("Something wrong", driver.getTitle());
    }

    @Test(enabled = false)
    public void ignoredTest() {
        driver.get("http:\\agiletestware.com");
    }

    @BumblebeeTestStep(name = "test three", description = "test three description", expected = "expect something")@Test(priority = 3)
    public void testThree() {
        throw new RuntimeException("You shall not pass!!!");
    }

    @AfterTest
    public void tearDown() {
        driver.quit();
    }
}

The example above produces the following test steps in HP ALM Test Plan:

Result in HP ALM

Capturing Screenshots

Selenium WebDriver tests failures are hard to debug if you only look at the stacktrace. Bumblebee can be configured to automatically capture screenshots for failed tests and send them to HP ALM. This is super useful for quick debugging.

Bumblebee Configuration

To enable screenshots, add the screenshotOnFailure element to the Bumblebee configuration.

Example:

<?xml version="1.0"?>
<bumblebee>
    <!-- URL of the Bumblebee Server -->
    <bumblebee_url>http://server_name:port/bumblebee</bumblebee_url>
    <!-- URL of HP ALM Server -->
    <alm_url>http://server_name:port/qcbin</alm_url>
    <!-- Name of HP ALM User -->
    <alm_user>qcuser</alm_user>
    <!-- Encrypted password: please use http://server_name:port/bumblebee/password/encrypt to encrypt your plain text password  -->
    <alm_encrypted_pass>fd4OMOXLJjkMR6e64RJh3Q==</alm_encrypted_pass>
    <!-- HP ALM Domain -->
    <alm_domain>DEFAULT</alm_domain>
    <!-- HP ALM Project -->
    <alm_project>annotations_demo</alm_project>
    <!-- Asynchronous (offline) update -->
    <async_update>true</async_update>    
    <!-- Capture screenshot on WebDriver test failure -->
    <screenshotOnFailure>true</screenshotOnFailure>
</bumblebee>

JUnit Configuration

To enable screenshot capturing, add an instance of com.agiletestware.bumblebee.annotations.webdriver.BumblebeeRule class as a JUnit rule to the test class (just mark BumblebeeRule instance with org.junit.Rule annotation) and invoke the setTakeScreenshotOnFail(true) method.

Bumblebee also provides com.agiletestware.bumblebee.annotations.webdriver.BumblebeeRuleBuilder builder class for the creation and configuration of BumblebeeRule instance.

JUnit Example: Screenshot capturing with BumblebeeRule:

import java.io.File;
import org.junit.Assert;
import org.junit.BeforeClass;
import org.junit.Rule;
import org.junit.Test;
import org.openqa.selenium.chrome.ChromeDriver;
import com.agiletestware.bumblebee.annotations.Bumblebee;
import com.agiletestware.bumblebee.annotations.webdriver.BumblebeeRule;
import com.agiletestware.bumblebee.annotations.webdriver.BumblebeeRuleBuilder;

@Bumblebee(testplan = "Subject\\webdriver", testlab = "Root\\webdriver", testset = "screenshot")
public class WebDriverTest {

    @Rule
    public final BumblebeeRule rule = new BumblebeeRuleBuilder(() -> new ChromeDriver())
            .setTakeScreenshotOnFailure(true).build();

    @BeforeClass
    public static void beforeClass() {
        System.setProperty("webdriver.chrome.driver", new File("chromedriver.exe").getAbsolutePath());
    }

    @Test
    public void testPass() {
        rule.getWebDriver().get("http://agiletestware.com");
        Assert.assertEquals("Agiletestware - Software for QA and Development Tools", rule.getWebDriver().getTitle());
    }

    @Test
    public void testFail() {
        rule.getWebDriver().get("http://agiletestware.com");
        Assert.assertEquals("Something wrong", rule.getWebDriver().getTitle());
    }

}

Results in HP ALM:

Result in HP ALM

Note

You don't need to quit WebDriver instance after each test explicitly. It is automatically quit by BumblebeeRule. If you don't wish to quit driver after each test, simply set a custom com.agiletestware.bumblebee.annotations.webdriver.WebDriverCloseBehavior with setWebDriverCloseBehavior() method of builder.

TestNG Configuration

To capture screenshots on failed test, add com.agiletestware.bumblebee.annotations.testng.BumblebeeTestNGListener listener to the POM file. This is in addition to com.agiletestware.bumblebee.annotations.testng.BumblebeeTestNGReporter.

Example pom.xml:

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>com.agiletestware</groupId>
    <artifactId>bumblebee-demo</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <name>Dummy tests for bumblebee project</name>
    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>3.1</version>
                <inherited>true</inherited>
                <configuration>
                    <source>1.8</source>
                    <target>1.8</target>
                </configuration>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>2.19</version>
                <configuration>
                    <properties>
                        <property>
                            <name>listener</name>
                            <value>com.agiletestware.bumblebee.annotations.testng.BumblebeeTestNGReporter,com.agiletestware.bumblebee.annotations.testng.BumblebeeTestNGListener</value>
                        </property>
                    </properties>
                </configuration>
            </plugin>
        </plugins>
    </build>
    <dependencies>
        <dependency>
            <groupId>org.testng</groupId>
            <artifactId>testng</artifactId>
            <version>6.14.3</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>com.agiletestware</groupId>
            <artifactId>bumblebee-annotations</artifactId>
            <version>0.0.9</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.seleniumhq.selenium</groupId>
            <artifactId>selenium-server</artifactId>
            <version>3.0.1</version>
        </dependency>
    </dependencies>
    <repositories>
        <repository>
            <id>release.agiletestware.com</id>
            <url>http://public.maven.agiletestware.com.s3-website-us-west-2.amazonaws.com/release</url>
        </repository>       
    </repositories>
</project>

Implement com.agiletestware.bumblebee.annotations.webdriver.WebDriverTest interface

It is required for a test class to implement com.agiletestware.bumblebee.annotations.webdriver.WebDriverTest interface which contains only one method - getWebDriver(). This method returns a RemoteWebDriver instance which is used by test.

import org.openqa.selenium.remote.DesiredCapabilities;
import org.openqa.selenium.remote.RemoteWebDriver;
import org.testng.Assert;
import org.testng.annotations.AfterMethod;
import org.testng.annotations.BeforeMethod;
import org.testng.annotations.Test;

import com.agiletestware.bumblebee.annotations.BooleanValue;
import com.agiletestware.bumblebee.annotations.Bumblebee;
import com.agiletestware.bumblebee.annotations.webdriver.WebDriverTest;

@Bumblebee(testplan = "Subject\\webdriver", testlab = "Root\\webdriver", testset = "screenshot")
public class RemoteWebDriverTest implements WebDriverTest {

    private static final String USERNAME = "userName";
    private static final String AUTOMATE_KEY = "key";
    private static final String URL = "http://" + USERNAME + ":" +
            AUTOMATE_KEY + "@ondemand.saucelabs.com:80/wd/hub";
    private RemoteWebDriver webDriver;

    @BeforeMethod
    public void setUp() {       
        try {
            webDriver = new RemoteWebDriver(new java.net.URL(URL), DesiredCapabilities.firefox());
        } catch (final Exception e) {
            throw new RuntimeException(e);
        }
    }

    @Test
    public void testOne() {
        webDriver.get("http://agiletestware.com");
        Assert.assertEquals(webDriver.getTitle(), "Agiletestware - Software for QA and Development Tools");
    }

    @Test
    public void testTwo() {
        webDriver.get("http://agiletestware.com");
        Assert.assertEquals(webDriver.getTitle(), "Something wrong");
    }

    @AfterMethod
    public void after() {
        webDriver.quit();
    }

    @Override
    public RemoteWebDriver getWebDriver() {
        return webDriver;
    }
}

Results in HP ALM:
Result in HP ALM