Docs
Bumblebee
Test Frameworks
JUnit

JUnit and ALM Integration

This setup requires a little bit of code adjustment. You should only do this is if there is a need to map java tests to alm tests. If you need simple upload of test results, use the Jenkins plugin.

JUnit along with TestNG are the most widely used test frameworks for writing unit and functional tests using a variety of testing tools like Selenium, Playwright, REST Assured, etc.

Bumblebee provides JUNIT annotations package that allows users to tag their Junit tests with ALM specific metadata and map JAVA tests to ALM tests in a variety of flexible workflows. When these mapped tests are run via CI or via the IDE, Bumblebee takes care of uploading both the tests and the results automatically. This setup is incredibly useful for teams that use ALM for test case management but are using open-source tools like Selenium for test automation and saves hours of work.

Configuration

Bumblebee server has already been installed and configured per instructions.

This example is for a Maven JUnit project

Bumblebee server and clients provide a rich set of configurations to help users create appropriate workflows. This section goes over a variety of configuration schemes.

  1. Modify the project pom.xml file and add the bumblebee-annotations dependency
<dependencies>
  <dependency>
      <groupId>com.agiletestware</groupId>
      <artifactId>bumblebee-annotations</artifactId>
      <version>0.1.11</version>
      <scope>test</scope>
   </dependency>
</dependencies>  	
  1. Configure Maven surefire plugin:
<build>
	<plugins>   
		<plugin>
			<groupId>org.apache.maven.plugins</groupId>
			<artifactId>maven-surefire-plugin</artifactId>
			<version>2.19</version>
			<configuration>
				<properties>
					<property>
						<name>listener</name>
						<value>com.agiletestware.bumblebee.annotations.BumblebeeJUnitListener</value>
					</property>
				</properties>
			</configuration>
		</plugin>
	</plugins>
</build>
  1. Add bumblebee_config.xml to the project root folder
<?xml version="1.0"?>
<bumblebee>
  <!-- URL of the Bumblebee Server -->
  <bumblebee_url>http://server_name:port/bumblebee</bumblebee_url>
  <!-- URL of HP ALM Server -->
  <alm_url>http://server_name:port/qcbin</alm_url>
  <!-- Name of HP ALM User -->
  <alm_user>qcuser</alm_user>
  <!-- Encrypted password: please use http://server_name:port/bumblebee/password/encrypt to encrypt your plain text password  -->
  <alm_encrypted_pass>fd4OMOXLJjkMR6e64RJh3Q==</alm_encrypted_pass>
  <!-- HP ALM Domain -->
  <alm_domain>DEFAULT</alm_domain>
  <!-- HP ALM Project -->
  <alm_project>annotations_demo</alm_project>
  <!-- Asynchronous (offline) update -->
  <async_update>false</async_update>
  <update_testplan>false</update_testplan>
  <enabled>false</enabled>
</bumblebee>

ALM Credentials in Server

Some security senstive users may not want to expose ALM credentials in the project configuration. Bumblebee allows users to specify ALM credentials within the Bumblebee Server in the alm-mappings.xml configuraiton section.

When specifying ALM credentials on the Bumblebee server side, they can be specified at project, domain or URL level as shown in the following examples.

Server side ALM credentials at project level can be specified as follows:

<?xml version="1.0" encoding="UTF-8"?>
 
<mappings>
    <url url="http://server_name:port/qcbin">
        <domain name="DEFAULT">
            <project name="annotations_demo">
                <credentials username="qcuser" encrypted_password="fd4OMOXLJjkMR6e64RJh3Q==" />
            </project>
        </domain>
    </url>
</mappings>

Server side ALM credentials at domain level can be specified as follows:

<?xml version="1.0" encoding="UTF-8"?>
 
<mappings>
    <url url="http://server_name:port/qcbin">
        <domain name="DEFAULT">
			<credentials username="qcuser" encrypted_password="fd4OMOXLJjkMR6e64RJh3Q==" />
            <project name="annotations_demo">
            </project>
        </domain>
    </url>
</mappings>

Server side ALM credentials at URL level can be specified as follows:

<?xml version="1.0" encoding="UTF-8"?>
 
<mappings>
    <url url="http://server_name:port/qcbin">
		<credentials username="qcuser" encrypted_password="fd4OMOXLJjkMR6e64RJh3Q==" />
        <domain name="DEFAULT">
            <project name="annotations_demo">
            </project>
        </domain>
    </url>
</mappings>

When ALM credentials are configured on the Bumblebee server side, the alm_user and alm_encrypted_pass attributes are not required on the client side bumblebee_config.xml file. However you may still specify these attributes, which will override the server side configuration.

Server side ALM credentials can also be specified to use API keys for authentication instead of username and password. To use this, create an ALM API key and specify it in the alm-mappings.xml as follows.

Server side ALM API keys at project level can be specified as follows:

<?xml version="1.0" encoding="UTF-8"?>
 
<mappings>
    <url url="http://server_name:port/qcbin">
        <domain name="DEFAULT">
            <project name="annotations_demo">
                <credentials username="qcuser" client_id="api-key-client-id" secret="api-key-secret" />
            </project>
        </domain>
    </url>
</mappings>

Server side ALM API keys at domain level can be specified as follows:

<?xml version="1.0" encoding="UTF-8"?>
 
<mappings>
    <url url="http://server_name:port/qcbin">
        <domain name="DEFAULT">
			<credentials username="qcuser" client_id="api-key-client-id" secret="api-key-secret" />
            <project name="annotations_demo">
            </project>
        </domain>
    </url>
</mappings>

Server side ALM API keys at URL level can be specified as follows:

<?xml version="1.0" encoding="UTF-8"?>
 
<mappings>
    <url url="http://server_name:port/qcbin">
		<credentials username="qcuser" client_id="api-key-client-id" secret="api-key-secret" />
        <domain name="DEFAULT">
            <project name="annotations_demo">
            </project>
        </domain>
    </url>
</mappings>

Note that the api-key-client-id and api-key-secret in the above examples should be replaced with the clientId and secret of the relevant ALM API key.

JAVA Properties

Values provided with Java Properties override the ones provided in XML configuration.

The Bumblebee properties can also be passed using java properties using us -D<property_name>

NameDescription
bumblebee.configPath to a configuration file. If not present, then ${project.basedir}/bumblebee_config.xml is used
bumblebee.config.bumblebee_urlURL of the Bumblebee Server
bumblebee.config.alm_urlURL of ALM Server
bumblebee.config.alm_userName of ALM User
bumblebee.config.alm_passwordEncrypted user password
bumblebee.config.alm_domainName of ALM Domain
bumblebee.config.alm_projectName of ALM Project
bumblebee.config.async_updateAsynchronous (offline) update
bumblebee.config.take_screenshotBoolean flag indicating whether a screenshot of a failure should be automatically done (for WebDriver tests only)
bumblebee.config.enabledBoolean flag indicating whether Bumblebee functionality is enabled or not
bumblebee.config.update_testplanBoolean flag indicating whether Bumblebee should update existing tests in ALM TestPlan module

Example

mvn clean test -Dbumblebee.config.bumblebee_url=https://company.alm.com:8080/bumblebee

Disable Exporting

To disable exporting all test results into HP ALM, add <enabled>false</enabled> into bumblebee_config.xml

<?xml version="1.0"?>
	<bumblebee>
	  <!-- URL of the Bumblebee Server -->
	  <bumblebee_url>http://server_name:port/bumblebee</bumblebee_url>
	  <!-- URL of HP ALM Server -->
	  <alm_url>http://server_name:port/qcbin</alm_url>
	  <!-- Name of HP ALM User -->
	  <alm_user>qcuser</alm_user>
	  <!-- Encrypted password: please use http://server_name:port/bumblebee/password/encrypt to encrypt your plain text password  -->
	  <alm_encrypted_pass>fd4OMOXLJjkMR6e64RJh3Q==</alm_encrypted_pass>
	  <!-- HP ALM Domain -->
	  <alm_domain>DEFAULT</alm_domain>
	  <!-- HP ALM Project -->
	  <alm_project>annotations_demo</alm_project>
	  <enabled>false</enabled>
	</bumblebee>

During the execution the following message will be shown in console output: Bumblebee: Bumblebee has been disabled in configuration --> Results will not be exported to HP ALM

Disable TestPlan Update

Sometimes users may not want to update existing tests in the TestPlan and only want to update test results in the TestLab. To do this, add the <update_testplan>false</update_testplan> configuration switch. By default (true), Bumblebee updates test in TestPlan for each run.

<bumblebee>
	<bumblebee_url>http://bbe:8888/bumblebee</bumblebee_url>
	<alm_url>http://alm:8080/qcbin</alm_url>
	<alm_user>user</alm_user>
	<alm_encrypted_pass>fd4OMOXLJjkMR6e64RJh3Q==</alm_encrypted_pass>
	<alm_domain>DEFAULT</alm_domain>
	<alm_project>demo</alm_project>
	<update_testplan>false</update_testplan>
</bumblebee>

In such a case, Bumblebee will not update existing tests in TestPlan and add a new Comments sections into a run in TestLab:

Result in HP ALM

while existing test steps will get No Run status:

Result in HP ALM

Override Server Configs

It is also possible to override server side configuration with the bumblebee XML configuration. To do that, just add a new set of mapping elements into configuration:

<bumblebee>
	<bumblebee_url>http://bbe:8888/bumblebee</bumblebee_url>
	<alm_url>http://alm:8080/qcbin</alm_url>
	<alm_user>user</alm_user>
	<alm_encrypted_pass>fd4OMOXLJjkMR6e64RJh3Q==</alm_encrypted_pass>
	<alm_domain>DEFAULT</alm_domain>
	<alm_project>demo</alm_project>
	
	<mapping field_type="test" field_label="user field" bumblebee_label="test user field" default_value="something"/>	
</bumblebee>

mapping element field_type has the following attributes:

  • field_type: a type of entity in ALM for which mapping is provided. The following are supported:
  • test: test in TestPlan
  • test-set: test set in TestLab
  • test-instance: test instance in TestLab
  • run-step: step of a test in TestPlan
  • step: step of a test in a run in TestLab
  • run: run in TestLab
  • field_label: label of a field in ALM (not a system name as in mapping configuration on server), e.g. 'Status' or 'User field', etc...
  • bumblebee_label: this is a label which should be used as @Parameter.name to pass a different value
  • default_value: default value

Example: If you wish to provide value for test's User field, the following element should be used:

<mapping field_type="test" field_label="user field" bumblebee_label="test user field" default_value="something"/>

When running Maven build, the following notification is displayed in the console output:

Bumblebee: Passing mappings configuration to the server: [Mapping [fieldLabel=user field, fieldType=TEST, defaultValue=something, bumblebeeLabel=test user field]]
Bumblebee: Mapping configuration on Bumblebee Server will be ignored	

And results in ALM:

Result in HP ALM

Disable specific tests

To disable exporting results for a particular test, specify the enabled = BooleanValue.FALSE attribute to Bumblebee annotation:

@Bumblebee(testlab = "Root\\junit_simple", testset = "class_annotations", testplan = "Subject\\junit_simple", enabled = BooleanValue.FALSE)
	public class SomePassSomeFailTest {
 
	  @Test
	  public void test1() {
	    Assert.fail("test1 failed");
	  }
	}

Add Bumblebee annotations

@BumblebeeJava annotation can be added on class or method level. Bumblebee annotation on method level overrides values set by Bumblebee annotation on class level

Bumblebee Annotation Reference

NameDescriptionApplicableRequired
testplanPath to test in a Test Plan in HP ALM. e.g. Subject\Test1Class, methodYes
testlabPath to test set in a test lab in HP ALM. e.g. Root\Test1Class, methodYes
testsetName of test set in HP ALMClass, methodYes
testnameIf specified, then the value of testname parameter will be set as the,name of test in HP ALM. If not set, test name will be set to fully,qualified method name, e.g. com.annotations.Demo.method1MethodNo
almidDefines id of a test in a HP ALM test plan which needs to be updated. If specified then testplan and testname are ignoredMethodNo
descriptionDescription for test in HP ALMMethodNo
overwriteAlmStepsIf set to BooleanValue.TRUE, all existing test steps will be deletedClass, methodNo
parametersAn array of custom parameters which are passed to the bumblebee server and then mapped to HP ALM custom fields. Please refer to documentation documentation on how to setup mappings on Bumblebee serverClass, methodNo

Setting Custom Fields

Bumblebee supports setting of custom fields for HP ALM Test, Test Set, Run, Design Step, Run Step, Test Instance

Bumblebee annotation allows users to pass custom parameters to Bumblebee Server which then are mapped to custom user fields in HP ALM.

Set the Parameter name to match the label attribute of mapping in mapping XML on the server. For example, if the server has this mapping: <mapping alm_field="TS_USER_01" label="user field"/>, the java code would look like with look up happening on the label which is more user friendly.

	public class UpdateExistingTest {
		@Bumblebee(testplan = "Subject\\custom fields", description = "Test description",
				testlab = "Root\\custom fields", testset = "map custom fields",
				parameters = {@Parameter(name = "user field", value = "value from JUnit") })
		@Test
		public void test1() {
			Assert.fail("test1 failed");
		}
	}

After export TS_USER_01 field of the test in HP ALM test plan gets the value from Java class:

Result in HP ALM

Dynamic Annotations

The following ALM parameters can contain dynamic values

  • testplan
  • testlab
  • testset
  • testname
  • description

To define a dynamic value, just add a placeholder in format: ${property_name}, where property_name is the name of property containing value for the parameter. Bumblebee resolves dynamic values in run time using Java System Properties or XML configuration file.

To set a value for dynamic parameter in bumblebee_config.xml file, just add new property element. E.g. to provide a value for $example placeholer

@Bumblebee(testset="Test set: ${example}")
<property name="example">some value</property>

To provide a value with Java System Property, just pass them into your Maven build

mvn clean test -Dexample=Something

If both XML configuration and Java System Property are provided, value in Java System Property will be used.

Attachments and Steps

Adding Attachments

Many times, tests need to output logs or screenshots or configs next to results to provide appropriate context. Bumblebee annotations package provides some convienent static utility class com.agiletestware.bumblebee.annotations.util.CurrentTest to help with this. The com.agiletestware.bumblebee.annotations.util.CurrentTest consist of a few methods

  • addAttachment(File): Add a single file attachment
  • addAttachment(String, InputStream): Add a single attachment from a stream
  • addAttachments(List<File>): Add list of files

Example of adding attachments:

@Bumblebee(testlab = "Root\\junit_simple", testset = "class_annotations", testplan = "Subject\\junit_simple")
public class SomePassSomeFailTest {
 
	@Test
	public void test1() {
		CurrentTest.addAttachment(new File("test.log"));
	}
}

Results in ALM:

Result in HP ALM

Adding Steps

Sometimes it is ncessary to programtically add design steps to ALM tests. Bumblebee provides com.agiletestware.bumblebee.annotations.util.CurrentTest.addStep(String) method which returns a builder object which provides convenient way to set attributes of step.

@Bumblebee(testlab = "Root\\junit_simple", testset = "class_annotations", testplan = "Subject\\junit_simple")
public class SomePassSomeFailTest {
 
	@Test
	public void test_additional_step() {
		final Step step = CurrentTest.addStep("Additional step")
				.description("description")
				.expected("expected")
				.actual("actual")
				.status(Status.FAILED)
				.build();
		step.addAttachment(new File("test.log"));
	}
}

Results in ALM:

Result in HP ALM

Log to Test Step

To add some log messages into test step's actual field, use com.agiletestware.bumblebee.annotations.util.CurrentTest.log(String) method:

import org.junit.Test;
 
import com.agiletestware.bumblebee.annotations.Bumblebee;
import com.agiletestware.bumblebee.annotations.util.CurrentTest;
 
@Bumblebee(testlab = "Root\\junit_simple", testset = "class_annotations", testplan = "Subject\\junit_simple")
public class SomePassSomeFailTest {
 
	@Test
	public void test1() {
		CurrentTest.log("Log line one");
		CurrentTest.log("Log line two");
	}
}

Results in ALM:

Result in HP ALM

Run Tests and Enjoy

Once the project has been configured with Bumblebee annoations, simply run via commandline, IDE, or CI and easily view the results in ALM.

Example Scenarios

We have created a few example mapping and configuration scenarios. There are a lot of possibilities due to the flexible nature of Bumblebee

Class Level Annotations

@Bumblebee(testlab = "Root\\junit_simple", testset = "class_annotations", testplan = "Subject\\junit_simple")
public class SomePassSomeFailTest {
 
  @Test
  public void test1() {
    Assert.fail("test1 failed");
  }
 
  @Test
  public void test2() {
    System.out.println("test2 passed");
  }
 
  @Ignore
  @Test
  public void test3() {
    System.out.println("test3 passed");
  }
 
  @Test
  public void test4() {
 
    throw new RuntimeException("test4 failed with exception");
  }
 
  @Test
  public void test5() {
    System.out.println("test5 passed");
  }
}

Results in ALM: Result in HP ALM

Method Level Annotations

	public class SomePassSomeFailTest {
 
		@Test
		public void test1() {
			Assert.fail("test1 failed");
		}
 
		@Bumblebee(testname="JUnit test",testlab = "Root\\junit_simple", testset = "method_annotation", testplan = "Subject\\junit_simple")
		@Test
		public void test2() {
			System.out.println("test2 passed");
		}
	}

Results in ALM: Result in HP ALM

Mapping Tests to ALM

Lets say we have some test in ALM with id=50 and we want to map it to our new java junit automation tests

Map to existing test

It contains two design steps:

Map to existing test

To map test method in java class to the test in HP ALM, almid property can be used. Also it is necessary to define testlab and testset properties :

	public class UpdateExistingTest {
		@Bumblebee(almid=50, description="Updated description", testlab = "Root\\junit existing test", testset = "map existing test")
		@Test
		public void test1() {
			Assert.fail("test1 failed");
		}
	}

Results in HP ALM: Test description in Test Plan is updated with the value of Bumblebee description field:

Result in HP ALM

There is also a new test step with name Test Results:

Result in HP ALM

And results are exported to the HP ALM Test Lab. In such a case, all steps get the same status:

Result in HP ALM

Bumblebee almid field can accept an array of test IDs almid={42,24}. In such a case, test result will be mapped into two tests in ALM.