Skip to content

NUnit and HP ALM Integration Guide

Description

Bumblebee provides a set of C# attributes which allows you to send your test reports directly from Visual Studio to HP ALM

Technically it consists of two parts:

  • BumblebeeTestFixture and BumblebeeTest attributes
  • XML configuration which is a part of your application configuration file.

Prerequisites

  • Agiletestware Bumblebee Server must be installed and be accessible via HTTP from user machine
  • Visual Studio 2012 or later with NUnit3TestAdapter installed
  • NuGet
  • NUnit 3

For running examples with Selenium WebDriver the following NuGet packages are needed:

  • Selenium.WebDriver
  • Selenium.WebDriver.ChromeDriver

Installation and configuration

1) Add "AgileTestware.Bumblebee.NUnit", "NUnit", "NUnit3TestAdapter" NuGet packages to your solution:

NuGet configuration

2) Add a new section and Bumblebee elements into your App.config file:

<configSections>
    <section name="Bumblebee"
             type="AgileTestware.Bumblebee.Core.Configurations.BumblebeeSection, AgileTestware.Bumblebee.Core"
             allowDefinition="Everywhere"
             allowLocation="true"/>  
</configSections>

<Bumblebee          
    bumblebee_url="http://bumblebee_server:port/bumblebee"          
    alm_url="http://alm_server:port/qcbin"
    alm_user="user"
    alm_encrypted_pass="fd4OMOXLJjkMR6e64RJh3Q=="
    alm_domain="DEFAULT"
    alm_project="project">
</Bumblebee>

Where

  • bumblebee_url - URL of Bumblebee Server
  • alm_url - URL of HP ALM server
  • alm_user - name of a user in HP ALM
  • alm_encrypted_pass - encrypted password for HP ALM user, please use http://bumblebee_server:port/bumblebee/password/encrypt to encrypt your plain text password
  • alm_domain - domain name in HP ALM
  • alm_project - project name in HP ALM

Add BumblebeeTestFixture and BumblebeeTest attributes to your NUnit test classes

In order to tell Bumblebee to export results of test class execution, that class needs to be marked with BumblebeeTestFixture attribute and all test methods shall be marked with BumblebeeTest attributes. If test method is not marked with BumblebeeTest attribute, its result will not be uploaded to HP ALM.

using AgileTestware.Bumblebee.NUnit.Attributes;
using NUnit.Framework;
using OpenQA.Selenium;
using OpenQA.Selenium.Chrome;
using OpenQA.Selenium.Support.UI;
using System;
using System.Collections.Generic;
using System.Collections.ObjectModel;

namespace Agiletstware.Demo.NUnit
{
    [BumblebeeTestFixture(TestPlan = "Subject\\Folder1\\Folder2", TestLab = "Root\\Folder1\\Folder2", TestSet = "Class attributes")]
    [TestFixture]
    public class ClassAttributesDemo
    {
        private IWebDriver webDriver;

        [SetUp]
        public void SetUp()
        {
            webDriver = new ChromeDriver();
        }

          [BumblebeeTest(Description ="Test that google returns Agiletestware.com as a first result for HP ALM and JUnit integration query")]
        [Test]
        public void SearchForTheBestIntegrationTool()
        {
            webDriver.Navigate().GoToUrl("http://google.com");
            IWebElement element = webDriver.FindElement(By.Name("q"));
            element.SendKeys("HP ALM and JUnit integration");
            element.Submit();
            IWebElement myDynamicElement = (new WebDriverWait(webDriver, TimeSpan.FromSeconds(1))
           .Until((d) =>
           {
               IWebElement el = d.FindElement(By.Id("resultStats"));
               if (el.Displayed && el.Enabled)
               {
                   return el;
               }
               return null;
           }));
            ReadOnlyCollection<IWebElement> findElements = webDriver.FindElements(By.XPath("//*[@id='rso']//h3/a"));
            Assert.True(findElements.Count > 0);
            Assert.True(findElements[0].GetAttribute("href").Contains("agiletestware"), "Ooops, we are not on the first place");
        }

        [BumblebeeTest]
        [Test]
        public void SomeFailingTest()
        {
            webDriver.Navigate().GoToUrl("http://google.com");
            webDriver.FindElement(By.LinkText("this is indeed a bad text for a link"));            
        }

        [TearDown]
        public void TearDown()
        {
            if (webDriver != null)
            {
                webDriver.Close();
                webDriver.Quit();
            }
        }
    }
}

BumblebeeTestFixture and BumblebeeTest attributes

BumblebeeTestFixture and BumblebeeTest attributes have the following properties:

Name Description Applicable Required
TestPlan Path to a test folder in HP ALM TestPlan where tests shall be created, e.g. Subject\Folder1\Folder2 Class, method Yes
TestLab Path to a test lab folder in HP ALM TestLab where test set shall be created, e.g. Root\Folder1\Folder2 Class, method Yes
TestSet Name of a test set in HP ALM TestLab to be created/updated Class, method Yes
TestName If specified, then the value of TestName parameter will be set as the,name of test in HP ALM. If not set, test name will be set to method name Method No
AlmId Defines id of a test in a HP ALM test plan which needs to be updated. If specified then TestPlan and TestName are ignored Method No
Description Description for test in HP ALM Method No
OverwriteAlmSteps If set to True, all existing test steps will be deleted Class, method No
Parameters JSON array of objects containing name and value for custom parameters which are passed to the bumblebee server and then mapped to HP ALM custom fields, e.g. [{'Name':'parameter 1','Value':'value 1'}, {'Name':'parameter 2','Value':'value 2'}]. Please refer to documentation documentation on how to setup mappings on Bumblebee server Class, method No
AlmReqIds An array of HP ALM requirements IDs to which this test should be linked Class, method No
AlmReqRecursive If set to True, test is linked to all child requirements of requirements defined y AlmReqIds parameter Class, method No

If the same property is defined on both class and method attribute, the method one takes precedence.

Exporting test results into HP ALM

Once your tests are marked with BumblebeeTestFixture and BumblebeeTest attributes, each time they are executed with NUnit adapter, results are automatically exported into HP ALM.

Examples

Export results of all test methods
using AgileTestware.Bumblebee.NUnit.Attributes;
using NUnit.Framework;
using OpenQA.Selenium;
using OpenQA.Selenium.Chrome;
using OpenQA.Selenium.Support.UI;
using System;
using System.Collections.Generic;
using System.Collections.ObjectModel;

namespace Agiletstware.Demo.NUnit
{
    [BumblebeeTestFixture(TestPlan = "Subject\\Folder1\\Folder2", TestLab = "Root\\Folder1\\Folder2", TestSet = "Class attributes")]
    [TestFixture]
    public class NunitDemoTest
    {
        private IWebDriver webDriver;

        [SetUp]
        public void SetUp()
        {
            webDriver = new ChromeDriver();
        }

        [BumblebeeTest]
        [Test]
        public void SearchForTheBestIntegrationTool()
        {
            webDriver.Navigate().GoToUrl("http://google.com");
            IWebElement element = webDriver.FindElement(By.Name("q"));
            element.SendKeys("HP ALM and JUnit integration");
            element.Submit();
            IWebElement myDynamicElement = (new WebDriverWait(webDriver, TimeSpan.FromSeconds(1))
           .Until((d) =>
           {
               IWebElement el = d.FindElement(By.Id("resultStats"));
               if (el.Displayed && el.Enabled)
               {
                   return el;
               }
               return null;
           }));
            ReadOnlyCollection<IWebElement> findElements = webDriver.FindElements(By.XPath("//*[@id='rso']//h3/a"));
            Assert.True(findElements.Count > 0);
            Assert.True(findElements[0].GetAttribute("href").Contains("agiletestware"), "Ooops, we are not on the first place");
        }

        [BumblebeeTest]
        [Test]
        public void SomeFailingTest()
        {
            webDriver.Navigate().GoToUrl("http://google.com");
            webDriver.FindElement(By.LinkText("this is indeed a bad text for a link"));            
        }

        [TearDown]
        public void TearDown()
        {
            if (webDriver != null)
            {
                webDriver.Close();
                webDriver.Quit();
            }
        }
    }
}

Results in HP ALM TestPlan:

Result in HP ALM

TestLab:

Result in HP ALM

Overriding class attributes on a particular method

Bumblebee allows you to override values defined in BumblebeeTestFixture for a single test method by setting values to BumblebeeTest attribute properties:

using AgileTestware.Bumblebee.NUnit.Attributes;
using NUnit.Framework;
using OpenQA.Selenium;
using OpenQA.Selenium.Chrome;
using OpenQA.Selenium.Support.UI;
using System;
using System.Collections.Generic;
using System.Collections.ObjectModel;

namespace Agiletstware.Demo.NUnit
{
    [BumblebeeTestFixture(TestPlan = "Subject\\Folder1\\Folder2", TestLab = "Root\\Folder1\\Folder2", TestSet = "Class attributes")]
    [TestFixture]
    public class NunitDemoTest
    {
        private IWebDriver webDriver;

        [SetUp]
        public void SetUp()
        {
            webDriver = new ChromeDriver();
        }

        [BumblebeeTest(TestName = "Overridden test name", Description = "Test that google returns Agiletestware.com as a first result for HP ALM and JUnit integration query")]
        [Test]
        public void SearchForTheBestIntegrationTool()
        {
            webDriver.Navigate().GoToUrl("http://google.com");
            IWebElement element = webDriver.FindElement(By.Name("q"));
            element.SendKeys("HP ALM and JUnit integration");
            element.Submit();
            IWebElement myDynamicElement = (new WebDriverWait(webDriver, TimeSpan.FromSeconds(1))
           .Until((d) =>
           {
               IWebElement el = d.FindElement(By.Id("resultStats"));
               if (el.Displayed && el.Enabled)
               {
                   return el;
               }
               return null;
           }));
            ReadOnlyCollection<IWebElement> findElements = webDriver.FindElements(By.XPath("//*[@id='rso']//h3/a"));
            Assert.True(findElements.Count > 0);
            Assert.True(findElements[0].GetAttribute("href").Contains("agiletestware"), "Ooops, we are not on the first place");
        }

        [BumblebeeTest]
        [Test]
        public void SomeFailingTest()
        {
            webDriver.Navigate().GoToUrl("http://google.com");
            webDriver.FindElement(By.LinkText("this is indeed a bad text for a link"));            
        }

        [TearDown]
        public void TearDown()
        {
            if (webDriver != null)
            {
                webDriver.Close();
                webDriver.Quit();
            }
        }
    }
}

Results in HP ALM TestPlan:

Result in HP ALM

TestLab:

Result in HP ALM

Setting custom fields in HP ALM from NUnit class

BumblebeeTestFixture and BumblebeeTest attributes allow users to pass custom parameters to Bumblebee Server which then are mapped to custom user fields in HP ALM. Please refer to documentation on how to setup mappings on Bumblebee server. Parameter name shall match to the "label" attribute of mapping in mapping XML on the server:

<mapping alm_field="TS_USER_01" label="user field"/>

To map some value to TS_USER_01 field in HP ALM, define a parameter with name="user field":

using AgileTestware.Bumblebee.NUnit.Attributes;
using NUnit.Framework;
using OpenQA.Selenium;
using OpenQA.Selenium.Chrome;
using OpenQA.Selenium.Support.UI;
using System;
using System.Collections.Generic;
using System.Collections.ObjectModel;

namespace Agiletstware.Demo.NUnit
{
    [BumblebeeTestFixture(TestPlan = "Subject\\Folder1\\Folder2", TestLab = "Root\\Folder1\\Folder2", TestSet = "Class attributes", Parameters = "[{'Name':'user field','Value':'some value'}]")]
    [TestFixture]
    public class NunitDemoTest
    {
        private IWebDriver webDriver;

        [SetUp]
        public void SetUp()
        {
            webDriver = new ChromeDriver();
        }

        [BumblebeeTest]
        [Test]
        public void SearchForTheBestIntegrationTool()
        {
            webDriver.Navigate().GoToUrl("http://google.com");
            IWebElement element = webDriver.FindElement(By.Name("q"));
            element.SendKeys("HP ALM and JUnit integration");
            element.Submit();
            IWebElement myDynamicElement = (new WebDriverWait(webDriver, TimeSpan.FromSeconds(1))
           .Until((d) =>
           {
               IWebElement el = d.FindElement(By.Id("resultStats"));
               if (el.Displayed && el.Enabled)
               {
                   return el;
               }
               return null;
           }));
            ReadOnlyCollection<IWebElement> findElements = webDriver.FindElements(By.XPath("//*[@id='rso']//h3/a"));
            Assert.True(findElements.Count > 0);
            Assert.True(findElements[0].GetAttribute("href").Contains("agiletestware"), "Ooops, we are not on the first place");
        }

        [TearDown]
        public void TearDown()
        {
            if (webDriver != null)
            {
                webDriver.Close();
                webDriver.Quit();
            }
        }
    }
}

After export TS_USER_01 field of the test in HP ALM test plan gets the value from your class:

Result in HP ALM

Currently Bumblebee supports setting of custom fields for HP ALM Test, Test Set and Run.

You can also override Parameters property for individual test method by setting its value in corresponding BumblebeeTest attribute.

Mapping tests to HP ALM requirements

Bumblebee annotations allows users to map their test classes/methods to HP ALM requirements with a help of two parameters: * AlmReqIds - an array of requirements IDs in HP ALM * AlmReqRecursive - defines if test should be also linked to child requirements

e.g.:

using AgileTestware.Bumblebee.NUnit.Attributes;
using NUnit.Framework;
using OpenQA.Selenium;
using OpenQA.Selenium.Chrome;
using OpenQA.Selenium.Support.UI;
using System;
using System.Collections.Generic;
using System.Collections.ObjectModel;

namespace Agiletstware.Demo.NUnit
{
    [BumblebeeTestFixture(TestPlan = "Subject\\Folder1\\Folder2", TestLab = "Root\\Folder1\\Folder2", TestSet = "Class attributes", AlmReqIds = new int[] {1, 3}, AlmReqRecursive = true)]
    [TestFixture]
    public class NunitDemoTest
    {
        private IWebDriver webDriver;

        [SetUp]
        public void SetUp()
        {
            webDriver = new ChromeDriver();
        }

        [BumblebeeTest ]
        [Test]
        public void SearchForTheBestIntegrationTool()
        {
            webDriver.Navigate().GoToUrl("http://google.com");
            IWebElement element = webDriver.FindElement(By.Name("q"));
            element.SendKeys("HP ALM and JUnit integration");
            element.Submit();
            IWebElement myDynamicElement = (new WebDriverWait(webDriver, TimeSpan.FromSeconds(1))
           .Until((d) =>
           {
               IWebElement el = d.FindElement(By.Id("resultStats"));
               if (el.Displayed && el.Enabled)
               {
                   return el;
               }
               return null;
           }));
            ReadOnlyCollection<IWebElement> findElements = webDriver.FindElements(By.XPath("//*[@id='rso']//h3/a"));
            Assert.True(findElements.Count > 0);
            Assert.True(findElements[0].GetAttribute("href").Contains("agiletestware"), "Ooops, we are not on the first place");
        }

        [TearDown]
        public void TearDown()
        {
            if (webDriver != null)
            {
                webDriver.Close();
                webDriver.Quit();
            }
        }
    }
}

After running tests, Bumblebee creates test in HP ALM and maps it to the requirements with ID 1 and 3, and also to their children:

Result in HP ALM

If requirement mappings are specified on a class-level annotation, they will be applied to all test methods of the class.

Map NUnit test method to existing test in HP ALM:

Here is the existing Manual test in HP ALM Test Plan with id = 16:

Map to existing test

It contains two design steps:

Map to existing test

To map test method in C# class to the test in HP ALM, AlmId property can be used. Also it is necessary to define TestLab and TestSet properties :

using AgileTestware.Bumblebee.NUnit.Attributes;
using NUnit.Framework;
using OpenQA.Selenium;
using OpenQA.Selenium.Chrome;
using OpenQA.Selenium.Support.UI;
using System;
using System.Collections.Generic;
using System.Collections.ObjectModel;

namespace Agiletstware.Demo.NUnit
{
    [BumblebeeTestFixture(TestPlan = "Subject\\Folder1\\Folder2", TestLab = "Root\\Folder1\\Folder2", TestSet = "Demo set")]
    [TestFixture]
    public class ClassAttributesDemo
    {
        private IWebDriver webDriver;

        [SetUp]
        public void SetUp()
        {
            webDriver = new ChromeDriver();
        }

        [BumblebeeTest(AlmId = 16, Description = "Test that google returns Agiletestware.com as a first result for HP ALM and JUnit integration query")]
        [Test]
        public void SearchForTheBestIntegrationTool()
        {
            webDriver.Navigate().GoToUrl("http://google.com");
            IWebElement element = webDriver.FindElement(By.Name("q"));
            element.SendKeys("HP ALM and JUnit integration");
            element.Submit();
            IWebElement myDynamicElement = (new WebDriverWait(webDriver, TimeSpan.FromSeconds(1))
           .Until((d) =>
           {
               IWebElement el = d.FindElement(By.Id("resultStats"));
               if (el.Displayed && el.Enabled)
               {
                   return el;
               }
               return null;
           }));
            ReadOnlyCollection<IWebElement> findElements = webDriver.FindElements(By.XPath("//*[@id='rso']//h3/a"));
            Assert.True(findElements.Count > 0);
            Assert.True(findElements[0].GetAttribute("href").Contains("agiletestware"), "Ooops, we are not on the first place");
        }

        [TearDown]
        public void TearDown()
        {
            if (webDriver != null)
            {
                webDriver.Close();
                webDriver.Quit();
            }
        }
    }
}

Results in HP ALM: Test description in Test Plan is updated with the value of Bumblebee description field:

Result in HP ALM

There is also a new test step with name Test Results:

Result in HP ALM

And results are exported to the HP ALM Test Lab:

Result in HP ALM

Disabling export of results to HP ALM

Sometimes exporting of test results into HP ALM is not desired (e.g. during debugging tests by developers), so it can be easily switched off by modifying Bumblebee section in App.config file.
Just add isEnabled="false" attribute to Bumblebee section and save App.config file.
e.g.:

<Bumblebee          
    bumblebee_url="http://bumblebee_server:port/bumblebee"          
    alm_url="http://alm_server:port/qcbin"
    alm_user="user"
    alm_encrypted_pass="fd4OMOXLJjkMR6e64RJh3Q=="
    alm_domain="DEFAULT"
    alm_project="project"
    isEnabled="false">
</Bumblebee>

During test execution you will get the following record in test output: Skipping HP ALM update. Bumblebee is disabled.

Adding attachments to test run in HP ALM (since 1.1.3)

In some cases it could be beneficial to have some test data like screenshots or logs accessible in HP ALM.

To add an arbitrary attachment to your test results in HP ALM you can use AgileTestware.Bumblebee.NUnit.CurrentTest class which provides AddAttachment method.
AddAttachment method takes an instance of AgileTestware.Bumblebee.Core.Attachment as a parameter and Attachment class provides several convenient static methods for creating attachments:

  • FromFile - creates an attachments from some local file
  • FromStream - creates attachments from Stream
  • FromByteArray - creates attachments from byte array

Here is a code snippet for adding some attachment to HP ALM:

using AgileTestware.Bumblebee.Core;
using AgileTestware.Bumblebee.NUnit;
using AgileTestware.Bumblebee.NUnit.Attributes;
using NUnit.Framework;
using System.Text;

namespace Agiletestware.Demo.NUnit
{
    [TestFixture]
    [BumblebeeTestFixture(TestPlan = "Subject\\NUnit\\Attachments", TestLab = "Root\\NUnit", TestSet = "Attachments Demo")]
    class SimpleAttachmentsDemo
    {
        [BumblebeeTest]
        [Test]
        public void AddSomeAttachmentTest()
        {
            //do something
            CurrentTest.AddAttachment(Attachment.FromByteArray(Encoding.ASCII.GetBytes("Some content"), "attachment.txt", "text/plain"));
            //do something
        }
    }
}

Results in HP ALM:

Result in HP ALM!

Adding test steps to HP ALM test (since 1.1.4)

Bumblebee provides functionality for adding test steps to HP ALM programmatically from test code.
By default, Bumblebee creates one synthetic test step with "Test Result" name which reflects execution status of NUnit test method. To add more steps, just call AddTestStep method of AgileTestware.Bumblebee.NUnit.CurrentTest class:

using AgileTestware.Bumblebee.Core;
using AgileTestware.Bumblebee.NUnit;
using AgileTestware.Bumblebee.NUnit.Attributes;
using NUnit.Framework;
using System.Text;

namespace Agiletestware.Demo.NUnit
{
    [TestFixture]
    [BumblebeeTestFixture(TestPlan = "Subject\\NUnit\\Steps", TestLab = "Root\\NUnit", TestSet = "Steps Demo")]
    class AddStepsDemo
    {


        [BumblebeeTest(TestName = "Test steps", Description = "Steps demo", OverwriteAlmSteps =true)]
        [Test]
        public void TestFirstGoogleResultIsReallyAboutPizza()
        {                        
            TestStep step = CurrentTest.AddTestStep("Step One", "Description for the first step", AgileTestware.Bumblebee.Core.Enums.TestCaseStatus.Failed);
            step.Expected = "Expected";
            step.Actual = "Actual";
            step.Error = "Some error";
            //Add attachment                
            step.Attachments.Add(Attachment.FromByteArray(Encoding.ASCII.GetBytes("Some content"), "attachment.txt", "text/plain"));

            CurrentTest.AddTestStep("Step Two", "Description for the second step", AgileTestware.Bumblebee.Core.Enums.TestCaseStatus.Passed);                                    
        }

    }
}

Results in HP ALM:

Result in HP ALM