Selenium WebDriver + HP ALM Integration Guide¶
Selenium WebDriver is one of the most widely used open source tool for automating browser interaction and is used by countless companies.
Bumblebee's JUnit and TestNG solution allows you to easily integrate Selenium WebDriver tests with HP ALM, map them to requirements, and automatically upload tests, and test results to HP ALM Testplan, TestLab, and Requirements Module.
Install Bumblebee Server before configuring maven projects
Bumblebee's JUnit and TestNG solution communicates with HP ALM via the Bumblebee server. You must install Bumblebee server before configuring your maven project. Bumblebee server setup instructions
Project Setup¶
Maven¶
Modify your pom.xml file and add agiletestware repository and bumblebee-annotations dependency:
<dependencies> <dependency> <groupId>com.agiletestware</groupId> <artifactId>bumblebee-annotations</artifactId> <version>0.0.6</version> <scope>test</scope> </dependency> </dependencies> <repositories> <repository> <id>com.agiletestware</id> <name>Agiletestware Maven Repository</name> <url>http://www.agiletestware.com/maven/</url> </repository> </repositories>
JUnit Maven surefire plugin config:
<build> <plugins> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-surefire-plugin</artifactId> <version>2.19</version> <configuration> <properties> <property> <name>listener</name> <value>com.agiletestware.bumblebee.annotations.BumblebeeJUnitListener</value> </property> </properties> </configuration> </plugin> </plugins> </build>
TestNG Maven surefire plugin config:
<build> <plugins> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-surefire-plugin</artifactId> <version>2.19</version> <configuration> <properties> <property> <name>listener</name> <value>com.agiletestware.bumblebee.annotations.testng.BumblebeeTestNGReporter</value> </property> </properties> </configuration> </plugin> </plugins> </build>
Configuration file¶
Bumblebee configuration file contains information about your HP ALM setup. The configuration values are read by Bumblebee's JUnit and TestNG solution and used to update test results in HP ALM.
Create bumblebee_config.xml file and put into the project root folder. Example:
<?xml version="1.0"?> <bumblebee> <!-- URL of the Bumblebee Server --> <bumblebee_url>http://server_name:port/bumblebee</bumblebee_url> <!-- URL of HP ALM Server --> <alm_url>http://server_name:port/qcbin</alm_url> <!-- Name of HP ALM User --> <alm_user>qcuser</alm_user> <!-- Encrypted password: please use http://server_name:port/bumblebee/password/encrypt to encrypt your plain text password --> <alm_encrypted_pass>fd4OMOXLJjkMR6e64RJh3Q==</alm_encrypted_pass> <!-- HP ALM Domain --> <alm_domain>DEFAULT</alm_domain> <!-- HP ALM Project --> <alm_project>annotations_demo</alm_project> <!-- Asynchronous (offline) update --> <async_update>true</async_update> </bumblebee>
Bumblebee Annotations¶
Bumblebee provides specialized annotations package that allows users to map JAVA tests to ALM tests and export their test results. @Bumblebee Java annotation can be added on JUnit/TestNG class or method level.
Annotation parameters¶
Name | Description | Applicable | Required |
---|---|---|---|
testplan | Path to test in a Test Plan in HP ALM. e.g. Subject\Test1 | Class, method | Yes |
testlab | Path to test set in a test lab in HP ALM. e.g. Root\Test1 | Class, method | Yes |
testset | Name of test set in HP ALM | Class, method | Yes |
testname | If specified, then the value of testname parameter will be set as the,name of test in HP ALM. If not set, test name will be set to fully,qualified method name, e.g. com.annotations.Demo.method1 | Method | No |
almid | Defines id of a test in a HP ALM test plan which needs to be updated. If specified then testplan and testname are ignored | Method | No |
description | Description for test in HP ALM | Method | No |
overwriteAlmSteps | If set to BooleanValue.TRUE, all existing test steps will be deleted | Class, method | No |
parameters | An array of custom parameters which are passed to the bumblebee server and then mapped to HP ALM custom fields. Please refer to documentation documentation on how to setup mappings on Bumblebee server | Class, method | No |
almReqIds | An array of HP ALM requirements IDs to which this test should be linked | Class, method | No |
almReqRecursive | If set to BooleanValue.TRUE, test is linked to all child requirements of requirements defined y almReqIds parameter | Class, method | No |
hasSteps | Works with TestNG only If set to BooleanValue.TRUE, test class is exported as a single test in HP ALM, and all its test methods are exported as test steps within the test | No |
Bumblebee annotation on method level overrides values set by Bumblebee annotation on class level.
To run JUnit/TestNG tests and send results to HP ALM, simply run Maven test phase on your project: Example: mvn test
Note
Unless stated, all the examples below are for JUnit. To run the same example on TestNG, simply replace org.junit.Test with org.testng.annotations.Test Imports are omitted for brevity.
Class Level Annotations¶
In this example, we have added the annotations to the class. This means that all the tests within this class will inherit the same testplan, testlab, testset, etc values.
package com.agiletestware.bumblebee.dummytest; @Bumblebee(testplan = "Subject\\webdriver", testlab = "Root\\webdriver", testset = "class annotations") public class WebDriverTest { private WebDriver driver; @BeforeClass public static void beforeClass() { System.setProperty("webdriver.chrome.driver", new File("chromedriver.exe").getAbsolutePath()); } @Before public void setUp() { this.driver = new ChromeDriver(); } @Test public void testOne() { driver.get("http:\\agiletestware.com"); Assert.assertEquals("Agiletestware - Software for QA and Development Tools", driver.getTitle()); } @Test public void testTwo() { driver.get("http:\\agiletestware.com"); Assert.assertEquals("Something wrong", driver.getTitle()); } @Ignore@Test public void ignoredTest() { driver.get("http:\\agiletestware.com"); } @Test public void testThree() { throw new RuntimeException("You shall not pass!!!"); } @After public void tearDown() { driver.quit(); } }
When the above example is executed, the results will be automaticaly posted to HP ALM.
Method Level Annotations¶
In some situations, you may not want to export all the tests or you may want to group tests in different testplan, testlab, testsets. By applying Bumblebee annotations to methods, the user can have more granular control.
public class WebDriverTest { private WebDriver driver; @BeforeClass public static void beforeClass() { System.setProperty("webdriver.chrome.driver", new File("chromedriver.exe").getAbsolutePath()); } @Before public void setUp() { this.driver = new ChromeDriver(); } @Bumblebee(testname = "testOne", testplan = "Subject\\webdriver", testlab = "Root\\webdriver", testset = "method annotations")@Test public void testOne() { driver.get("http:\\agiletestware.com"); Assert.assertEquals("Agiletestware - Software for QA and Development Tools", driver.getTitle()); } @Test public void testTwo() { driver.get("http:\\agiletestware.com"); Assert.assertEquals("Something wrong", driver.getTitle()); } @Ignore@Test public void ignoredTest() { driver.get("http:\\agiletestware.com"); } @Test public void testThree() { throw new RuntimeException("You shall not pass!!!"); } @After public void tearDown() { driver.quit(); } }
When the above example is executed, only testOne() will be exported to HP ALM.
Overriding annotation values¶
In some situations, you may class-level annotations but may not want to apply them to all the methods. In such a situation, you can provide both class-level and method-level annotations to override the class-level values.
//class-level annotations values @Bumblebee(testplan = "Subject\\webdriver", testlab = "Root\\webdriver", testset = "overridden values") public class WebDriverTest { private WebDriver driver; @BeforeClass public static void beforeClass() { System.setProperty("webdriver.chrome.driver", new File("chromedriver.exe").getAbsolutePath()); } @Before public void setUp() { this.driver = new ChromeDriver(); } @Test public void testOne() { driver.get("http:\\agiletestware.com"); Assert.assertEquals("Agiletestware - Software for QA and Development Tools", driver.getTitle()); } //method-level annotations values @Bumblebee(testname = "test two", testplan = "Subject\\webdriver", testlab = "Root\\webdriver", testset = "overridden values")@Test public void testTwo() { driver.get("http:\\agiletestware.com"); Assert.assertEquals("Something wrong", driver.getTitle()); } @Ignore@Test public void ignoredTest() { driver.get("http:\\agiletestware.com"); } @Test public void testThree() { throw new RuntimeException("You shall not pass!!!"); } @After public void tearDown() { driver.quit(); } }
When the above example is executed, testOne() and testThree() will use class-level annotation values. testTwo() will use method-level annotation values.
Map test method to existing test in HP ALM¶
In many companies, it is very common for business analysts or manual test engineers write manual tests in HP ALM. These manual tests are then automated at a later stage. Bumblebee annotation can be used to help implement this workflow. As tests are automated in JAVA, they can be mapped to the manual tests in HP ALM.
For Example: Here is the existing test in HP ALM Test Plan with id = 112
It contains three design steps:
To map this manual ALM test to a test method in JAVA, use the Bumblebee annotation's almid parameter along with the testlab and testset values.
@Bumblebee(testplan = "Subject\\webdriver", testlab = "Root\\webdriver", testset = "overridden values") public class WebDriverTest { private WebDriver driver; @BeforeClass public static void beforeClass() { System.setProperty("webdriver.chrome.driver", new File("chromedriver.exe").getAbsolutePath()); } @Before public void setUp() { this.driver = new ChromeDriver(); } @Test public void testOne() { driver.get("http:\\agiletestware.com"); Assert.assertEquals("Agiletestware - Software for QA and Development Tools", driver.getTitle()); } @Bumblebee(almid = 112, testlab = "Root\\webdriver", testset = "overridden values", description = "description")@Test public void testTwo() { driver.get("http:\\agiletestware.com"); Assert.assertEquals("Something wrong", driver.getTitle()); } @Ignore@Test public void ignoredTest() { driver.get("http:\\agiletestware.com"); } @Test public void testThree() { throw new RuntimeException("You shall not pass!!!"); } @After public void tearDown() { driver.quit(); } }
In the example above, testTwo() has been mapped to the manual testcase in ALM and also updated the test description using the description annotation parameter.
There is also a new test step with name Test Results:
And results are exported to the HP ALM Test Lab:
Custom Fields Configuration¶
Most HP ALM projects have some custom user-defined fields. If these fields are required, they must be specified by the user. Fortunately, Bumblebee annotation frameworks make it super easy to configure custom user fields. Bumblebee annotation framework allows users to pass custom parameters to Bumblebee Server which contains the mapping to the custom user fields in HP ALM.
Bumblebee server mapping format
This is the format for specifying ALM's custom user fields in the Bumblebee server configuration. In this example, TS_USER_01 is the actual custom field, ==label is the user friendly key that will be used within the JAVA annotations for mapping.
Please refer to ALM custom fields for detailed steps on how to setup mappings on Bumblebee server.
Example: Lets say we have a custom text field: TS_USER_01 field in HP ALM and we want to set this field to value from Java. Since we have already mapped this on Bumblebee server, we can use the label value user field in our JAVA annotation to complete the mapping.
@Bumblebee(testplan = "Subject\\webdriver", testlab = "Root\\webdriver", testset = "overridden values") public class WebDriverTest { private WebDriver driver; @BeforeClass public static void beforeClass() { System.setProperty("webdriver.chrome.driver", new File("chromedriver.exe").getAbsolutePath()); } @Before public void setUp() { this.driver = new ChromeDriver(); } @Bumblebee(almid = 112, testlab = "Root\\webdriver", testset = "overridden values", description = "description", parameters = {@Parameter(name = "user field", value = "value from Java") })@Test public void testTwo() { driver.get("http:\\agiletestware.com"); Assert.assertEquals("Something wrong", driver.getTitle()); } @After public void tearDown() { driver.quit(); } }
Once this test is executed, the mappings values are passed from the tests to the Bumblebee server and configured at run time.
Tip
Bumblebee annotations are very flexible and powerful and can be used to set custom fields for HP ALM Test, Test Set and Run.
HP ALM Requirements Mapping¶
Bumblebee annotations allows users to map their test classes/methods to HP ALM requirements module with a help of two parameters:
- almReqIds: an array of requirements IDs in HP ALM
- almReqRecursive: defines if test should be also linked to child requirements
Example:
@Bumblebee(testplan = "Subject\\webdriver", testlab = "Root\\webdriver", testset = "overridden values") public class WebDriverTest { private WebDriver driver; @BeforeClass public static void beforeClass() { System.setProperty("webdriver.chrome.driver", new File("chromedriver.exe").getAbsolutePath()); } @Before public void setUp() { this.driver = new ChromeDriver(); } @Bumblebee(testlab = "Root\\webdriver", testset = "overridden values", description = "description", almReqIds = { 1, 4 }, almReqRecursive = BooleanValue.TRUE)@Test public void testThree() { throw new RuntimeException("You shall not pass!!!"); } @After public void tearDown() { driver.quit(); } }
In the example above, Bumblebee creates tests in HP ALM and then maps it to the requirements with ID 1 and 4, and also to their children:
If requirement mappings are specified on a class-level annotation, they will be applied to all test methods of the class.
Exporting all test methods into one HP ALM test (TestNG only)¶
Sometimes tests classes are designed in such a way that each of them represents a single test case and test methods are just test steps. If you have such a test design, Bumblebee can help map this to HP ALM.
To export all methods of a single test class to HP ALM, mark the class with @Bumblebee
annotation with hasSteps
attribute set to BooleanValue.TRUE
.
Example:
@Bumblebee(testplan = "Subject\\webdriver", testlab = "Root\\webdriver", testset = "overridden values", hasSteps = BooleanValue.TRUE) public class WebDriverTest { private WebDriver driver; @BeforeClass public static void beforeClass() { System.setProperty("webdriver.chrome.driver", new File("chromedriver.exe").getAbsolutePath()); } @BeforeTest public void setUp() { this.driver = new ChromeDriver(); } @Test(priority = 1) public void testOne() { driver.get("http:\\agiletestware.com"); Assert.assertEquals("Agiletestware - Software for QA and Development Tools", driver.getTitle()); } @Test(priority = 2) public void testTwo() { driver.get("http:\\agiletestware.com"); Assert.assertEquals("Something wrong", driver.getTitle()); } @Test(enabled = false) public void ignoredTest() { driver.get("http:\\agiletestware.com"); } @Test(priority = 3) public void testThree() { throw new RuntimeException("You shall not pass!!!"); } @AfterTest public void tearDown() { driver.quit(); } }
In such a case, the new test with three steps will be created in HP ALM Test Plan:
and results will also appear in Test Lab:
Note
Bumblebee relies on test method execution order provided by TestNG, so you have to provide a fixed test methods execution order somehow (e.g. via priorities). Otherwise your test steps will have incorrect execution order and consequently wrong order in HP ALM.
By default, test step names are the same as names of corresponding test methods in your Java code. To change test step names and some other export settings, you can use the @BumblebeeTestStep
. This annotation can be applied to test methods only and has the following properties:
Name | Description | Required |
---|---|---|
name | Name of a test step in HP ALM | Yes |
description | Description of a test step in HP ALM | No |
expected | Value for Expected field of a test step in HP ALM |
No |
Example:
@Bumblebee(testplan = "Subject\\webdriver", testlab = "Root\\webdriver", testset = "overridden values", hasSteps = BooleanValue.TRUE) public class WebDriverTest { private WebDriver driver; @BeforeClass public static void beforeClass() { System.setProperty("webdriver.chrome.driver", new File("chromedriver.exe").getAbsolutePath()); } @BeforeTest public void setUp() { this.driver = new ChromeDriver(); } @Test(priority = 1) public void testOne() { driver.get("http:\\agiletestware.com"); Assert.assertEquals("Agiletestware - Software for QA and Development Tools", driver.getTitle()); } @Test(priority = 2) public void testTwo() { driver.get("http:\\agiletestware.com"); Assert.assertEquals("Something wrong", driver.getTitle()); } @Test(enabled = false) public void ignoredTest() { driver.get("http:\\agiletestware.com"); } @BumblebeeTestStep(name = "test three", description = "test three description", expected = "expect something")@Test(priority = 3) public void testThree() { throw new RuntimeException("You shall not pass!!!"); } @AfterTest public void tearDown() { driver.quit(); } }
The example above produces the following test steps in HP ALM Test Plan:
Capturing Screenshots¶
Selenium WebDriver tests failures are hard to debug if you only look at the stacktrace. Bumblebee can be configured to automatically capture screenshots for failed tests and send them to HP ALM. This is super useful for quick debugging.
Bumblebee Configuration
To enable screenshots, add the screenshotOnFailure element to the Bumblebee configuration.
Example:
<?xml version="1.0"?> <bumblebee> <!-- URL of the Bumblebee Server --> <bumblebee_url>http://server_name:port/bumblebee</bumblebee_url> <!-- URL of HP ALM Server --> <alm_url>http://server_name:port/qcbin</alm_url> <!-- Name of HP ALM User --> <alm_user>qcuser</alm_user> <!-- Encrypted password: please use http://server_name:port/bumblebee/password/encrypt to encrypt your plain text password --> <alm_encrypted_pass>fd4OMOXLJjkMR6e64RJh3Q==</alm_encrypted_pass> <!-- HP ALM Domain --> <alm_domain>DEFAULT</alm_domain> <!-- HP ALM Project --> <alm_project>annotations_demo</alm_project> <!-- Asynchronous (offline) update --> <async_update>true</async_update> <!-- Capture screenshot on WebDriver test failure --> <screenshotOnFailure>true</screenshotOnFailure> </bumblebee>
JUnit Configuration¶
To enable screenshot capturing, add an instance of com.agiletestware.bumblebee.annotations.webdriver.BumblebeeRule class as a JUnit rule to the test class (just mark BumblebeeRule instance with org.junit.Rule annotation) and invoke the setTakeScreenshotOnFail(true) method.
Bumblebee also provides com.agiletestware.bumblebee.annotations.webdriver.BumblebeeRuleBuilder builder class for the creation and configuration of BumblebeeRule instance.
JUnit Example: Screenshot capturing with BumblebeeRule:
import java.io.File; import org.junit.Assert; import org.junit.BeforeClass; import org.junit.Rule; import org.junit.Test; import org.openqa.selenium.chrome.ChromeDriver; import com.agiletestware.bumblebee.annotations.Bumblebee; import com.agiletestware.bumblebee.annotations.webdriver.BumblebeeRule; import com.agiletestware.bumblebee.annotations.webdriver.BumblebeeRuleBuilder; @Bumblebee(testplan = "Subject\\webdriver", testlab = "Root\\webdriver", testset = "screenshot") public class WebDriverTest { @Rule public final BumblebeeRule rule = new BumblebeeRuleBuilder(() -> new ChromeDriver()) .setTakeScreenshotOnFailure(true).build(); @BeforeClass public static void beforeClass() { System.setProperty("webdriver.chrome.driver", new File("chromedriver.exe").getAbsolutePath()); } @Test public void testPass() { rule.getWebDriver().get("http://agiletestware.com"); Assert.assertEquals("Agiletestware - Software for QA and Development Tools", rule.getWebDriver().getTitle()); } @Test public void testFail() { rule.getWebDriver().get("http://agiletestware.com"); Assert.assertEquals("Something wrong", rule.getWebDriver().getTitle()); } }
Results in HP ALM:
Note
You don't need to quit WebDriver instance after each test explicitly. It is automatically quit by BumblebeeRule. If you don't wish to quit driver after each test, simply set a custom com.agiletestware.bumblebee.annotations.webdriver.WebDriverCloseBehavior with setWebDriverCloseBehavior() method of builder.
TestNG Configuration¶
To capture screenshots on failed test, add com.agiletestware.bumblebee.annotations.testng.BumblebeeTestNGListener listener to the POM file. This is in addition to com.agiletestware.bumblebee.annotations.testng.BumblebeeTestNGReporter.
Example pom.xml:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> <modelVersion>4.0.0</modelVersion> <groupId>com.agiletestware</groupId> <artifactId>bumblebee-demo</artifactId> <version>0.0.1-SNAPSHOT</version> <name>Dummy tests for bumblebee project</name> <build> <plugins> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-compiler-plugin</artifactId> <version>3.1</version> <inherited>true</inherited> <configuration> <source>1.8</source> <target>1.8</target> </configuration> </plugin> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-surefire-plugin</artifactId> <version>2.19</version> <configuration> <properties> <property> <name>listener</name> <value>com.agiletestware.bumblebee.annotations.testng.BumblebeeTestNGReporter,com.agiletestware.bumblebee.annotations.testng.BumblebeeTestNGListener</value> </property> </properties> </configuration> </plugin> </plugins> </build> <dependencies> <dependency> <groupId>org.testng</groupId> <artifactId>testng</artifactId> <version>6.14.3</version> <scope>test</scope> </dependency> <dependency> <groupId>com.agiletestware</groupId> <artifactId>bumblebee-annotations</artifactId> <version>0.0.6</version> <scope>test</scope> </dependency> <dependency> <groupId>org.seleniumhq.selenium</groupId> <artifactId>selenium-server</artifactId> <version>3.0.1</version> </dependency> </dependencies> <repositories> <repository> <id>com.agiletestware</id> <name>Agiletestware Maven Repository</name> <url>http://www.agiletestware.com/maven/</url> <layout>default</layout> </repository> </repositories> </project>
Implement com.agiletestware.bumblebee.annotations.webdriver.WebDriverTest interface
It is required for a test class to implement com.agiletestware.bumblebee.annotations.webdriver.WebDriverTest interface which contains only one method - getWebDriver(). This method returns a RemoteWebDriver instance which is used by test.
import org.openqa.selenium.remote.DesiredCapabilities; import org.openqa.selenium.remote.RemoteWebDriver; import org.testng.Assert; import org.testng.annotations.AfterMethod; import org.testng.annotations.BeforeMethod; import org.testng.annotations.Test; import com.agiletestware.bumblebee.annotations.BooleanValue; import com.agiletestware.bumblebee.annotations.Bumblebee; import com.agiletestware.bumblebee.annotations.webdriver.WebDriverTest; @Bumblebee(testplan = "Subject\\webdriver", testlab = "Root\\webdriver", testset = "screenshot") public class RemoteWebDriverTest implements WebDriverTest { private static final String USERNAME = "userName"; private static final String AUTOMATE_KEY = "key"; private static final String URL = "http://" + USERNAME + ":" + AUTOMATE_KEY + "@ondemand.saucelabs.com:80/wd/hub"; private RemoteWebDriver webDriver; @BeforeMethod public void setUp() { try { webDriver = new RemoteWebDriver(new java.net.URL(URL), DesiredCapabilities.firefox()); } catch (final Exception e) { throw new RuntimeException(e); } } @Test public void testOne() { webDriver.get("http://agiletestware.com"); Assert.assertEquals(webDriver.getTitle(), "Agiletestware - Software for QA and Development Tools"); } @Test public void testTwo() { webDriver.get("http://agiletestware.com"); Assert.assertEquals(webDriver.getTitle(), "Something wrong"); } @AfterMethod public void after() { webDriver.quit(); } @Override public RemoteWebDriver getWebDriver() { return webDriver; } }