The CICD Dilemma

Agiletestware works with a variety of engineering organization all over the world. It is always enlightening to see how different organization implement quality engineering discipline and try to achieve CICD (continuous integration and continuous delivery). Our experience shows that CICD is a long journey and requires a great amount of experimentation, discipline, and custom integrations with a variety of tools and frameworks. There is no silver bullet that will get you to the promised land 😢 CICD takes time, patience, and a good vision 😇

Many of our customers use a wide variety of testing frameworks, tools, CI systems, ALM solution, bug trackers, requirement management systems, etc. All of them want to achieve CICD but integrating all these technologies is a very challenging task when you need to do your day job and ship features to customers.

In these series of blog posts, we will go over how many of our customers have come one step closer to achieving CICD. The blog posts will explain how a variety of testing frameworks and tools and have been integrated with HP ALM (our customer’s ALM software for tracking projects) using Agiletestware’s Bumblebeeproduct.

The typical technology landscape at most large companies

Before we explain the solution, let us explain the typical setup found in many of our customers. Hopefully, many of you will relate to this situation.

CI Applications – Jenkins, Bamboo, and TeamCity

CI system are at the center of achieving CICD. The three biggest and most mature CI applications are JenkinsAtlassian Bamboo, and Jetbrain’s TeamCity. CI systems are used to build software, run tests, and do a variety of other tasks. In mature engineering organizations, CI systems are used to deploy software to achieve CD. All the information about these various tasks (build, test results, etc) are contained within the CI application.

What is HP ALM?

HP ALM is the leading solution in the industry for tracking complex software project. HP ALM is used for bug tracking, requirements management, test case management, and much more. HP ALM is a commercial product and integrates with various other tools from HP tools for testing web, mobile, and various other software products.

Web Testing and Selenium WebDriver

Selenium Webdriver is an open source tool for automating browsers. Primarily, it is for automating web applications for testing purposes. Selenium has the support of some of the largest browser vendors who have taken (or are taking) steps to make Selenium a native part of their browser. It is also the core technology in countless other browser automation tools, APIs and frameworks. If you are not using Selenium for testing web applications, you should re-think your testing tool strategy and take a closer look at Selenium WebDriver 😀

Mobile Testing and Appium

Appium is an open source test automation framework used for testing native, hybrid and mobile web apps. Appium aims to automate ios, Android, and Windows mobile app from any modern programming language and any test framework. Appium is supported by SauceLabs (leading provider of web and mobile test cloud). Appium is most widely used mobile testing framework and is free.

JAVA Test Frameworks – testNG and JUnit

Testing tools like Selenium and Appium require some test framework. In the JAVA world, two of the most widely used test frameworks are JUnit and testNG.

JUnit: JUnit is a simple framework to write repeatable tests. It is an instance of the xUnit architecture for unit testing frameworks.

testNG: TestNG is a testing framework inspired from JUnit and NUnit but introducing some new functionalities that make it more powerful and easier to use.

All sounds simple. So what is the problem?

If you are a HP ALM shop and using HP tools (UFT, LeanFT, QTP, etc) for testing all your applications, then there are no issues and life should be good. Every software development and testing activity should nicely be reflected in HP ALM.

However, the reality is that most organizations are rarely a homogeneous HP shop. Some teams may be using HP Tools (UFT, LeanFT, QTP), mobile testing team may be using Appium or other similar open source tools, browser application testing teams may be using Selenium WebDriver and cloud browser solutions like SauceLabs and BrowserStack, API testing team may be using using ReadyAPI, and unit test is hopefully done via some x-unit framework like Junit or testNG and most of these activities are controlled via Jenkins/Bamboo/TeamCity.

The end result is that you end up with islands of information within your organization. Some information is in HP ALM and some is in Jenkins/Bamboo/Teamcity and it is impossible to get a unified view of the project. As a consequence, engineering management often makes an uninformed decision because they don’t know how to collect all the data. There are just too much human element, too many reports to prepare, and it is just a pain and no one wants to do this.

How can Agiletestware help?

Agiletestware has been helping engineering organizations achieve CICD through many of our tools. We can’t solve all your problems but we may be able to solve many of them to get you closer to CICD. HP ALM is a significant investment and you should get the most out of your investment. We believe HP ALM is still the best ALM solution in the market and has the most mature bug tracking, agile project management, requirements management, and test case management modules in a single solution.

Agiletestware’s Bumblebee product can help integrate CI applications like Jenkins/Bamboo/Teamcity, JAVA tests frameworks like testNG/JUnit, and web and mobile testing tools like Selenium WebDriver / Appium with HP ALM with little to no coding. Bumblebee simply intercepts activities within your existing testing tools and CI frameworks and exports it to HP ALM in real-time.


I am the founder of Agiletestware. I have been working as a Quality Engineering engineer, manager, architect, and consultant for about 10+ years in Los Angeles, and San Francisco. I have worked with a variety of test frameworks, test infrastructure automation strategies and tools, and various ALM and test case management applications. If you need advice on how to make your teams and processes efficient, transparent, and how best to integrate testing into your CICD strategy, shoot me an email and I would love to chat.

539 thoughts on “The CICD Dilemma”

  1. Pingback: Nike Element 87
  2. Pingback: jordan retro 12
  3. Pingback: nike air max
  4. Pingback: nike air max 90
  5. Pingback: Basketball Jersey
  6. Pingback: air jordans
  7. Pingback: jordan shoes
  8. Pingback: adidas zx flux
  9. Pingback: Balenciaga
  10. Pingback: moncler
  11. Pingback: Nike Outlet Online
  12. Pingback: Nike Outlet Store
  13. Pingback: golden goose
  14. Pingback: jordan shoes
  15. Pingback: yeezy
  16. Pingback: kyrie 3
  17. Pingback: adidas yeezy
  18. Pingback: nike air max
  19. Pingback: Yeezy 350
  20. Pingback: Yeezy 350
  21. Pingback: nike flyknit
  22. Pingback: yeezy 700
  23. Pingback: Nike Outlet Store
  24. Pingback: jordans
  25. Pingback: nike cortez
  26. Pingback: yeezy boost
  27. Pingback: Yeezy
  28. Pingback: Yeezys
  29. Pingback: off white hoodie
  30. Pingback: Adidas Yeezy
  31. Pingback: Yeezy
  32. Pingback: air max 270
  33. Pingback: adidas superstar
  34. Pingback: fila shoes
  35. Pingback: yeezy boost
  36. Pingback: air max
  37. Pingback: yeezy boost 350
  38. Pingback: jordan 12
  39. Pingback: golden goose
  40. Pingback: kd shoes
  41. Pingback: kd shoes
  42. Pingback: nike shox for men
  43. Pingback: supreme new york
  44. Pingback: kd shoes
  45. Pingback: nba jerseys
  46. Pingback: polo ralph lauren
  47. Pingback: nike air max
  48. Pingback: adidas yeezy
  49. Pingback: air max 97
  50. Pingback: adidas nmd
  51. Pingback: air max 97
  52. Pingback: timberland outlet
  53. Pingback: nike vapormax
  54. Pingback: yeezy boost
  55. Pingback: longchamp handbags
  56. Pingback: kyrie shoes
  57. Pingback: jimmy choo
  58. Pingback: nike air max 270
  59. Pingback: kobe sneakers
  60. Pingback: nike air max 2017
  61. Pingback: nike air max95
  62. Pingback: jordan shoes
  63. Pingback: hermes belt
  64. Pingback: yeezys
  65. Pingback: kyrie 5 shoes
  66. Pingback: balenciaga speed
  67. Pingback: ferragamo belt
  68. Pingback: nike air vapormax
  69. Pingback: yeezy boost 350 v2
  70. Pingback: yeezys
  71. Pingback: air jordan
  72. Pingback: jordan shoes
  73. Pingback: golden goose
  74. Pingback: coach outlet store
  75. Pingback: louboutin outlet
  76. Pingback: coach outlet
  77. Pingback: nike shoes
  78. Pingback: cheap jordans
  79. Pingback: moncler outlet
  80. Pingback: yeezy boost 350 v2
  81. Pingback: nike 270
  82. Pingback: nike air max
  83. Pingback: air jordans
  84. Pingback: mbt
  85. Pingback: lebron 15
  86. Pingback: supreme shirt
  87. Pingback: kyrie 5
  88. Pingback: hermes
  89. Pingback: off white clothing
  90. Pingback: MLB Jerseys
  91. Pingback: lebron 17
  92. Pingback: longchamp handbags
  93. Pingback: hermes birkin
  94. Pingback: rock and blue kort
  95. Pingback: air jordan
  96. Pingback: louboutin
  97. Pingback: jordan retro
  98. Pingback: converse outlet
  99. Pingback: coach outlet
  100. Pingback: dainese pro armor
  101. Pingback: vans shoes
  102. Pingback: adidas tubular x
  103. Pingback: yeezy
  104. Pingback: yeezy boost 350
  105. Pingback: curry shoes
  106. Pingback: supreme clothing
  107. Pingback: kd shoes
  108. Pingback: chinese 12s
  109. Pingback: grote maten h m
  110. Pingback: kyrie 6
  111. Pingback: yeezy boost 350
  112. Pingback: koti wale lehenga
  113. Pingback: supreme outlet
  114. Pingback: sautoir pas cher
  115. Pingback: chaussure clou
  116. Pingback: kd 12
  117. Pingback: golden goose
  118. Pingback: nike shox white
  119. Pingback: chrome hearts
  120. Pingback: hermes birkin
  121. Pingback: zulily harem pants
  122. Pingback: irving shoes
  123. Pingback: jordan retro
  124. Pingback: nike air max
  125. Pingback: pure boost red
  126. Pingback: huawei honor 8 pro
  127. Pingback: casual ecco o2 178
  128. Pingback: goyard bags
  129. Pingback: longchamp handbags
  130. Pingback: kyrie irving shoes
  131. Pingback: nike air max 2018
  132. Pingback: nike air max
  133. Pingback: convese outlet
  134. Pingback: nike air max 90
  135. Pingback: off white
  136. Pingback: high neck frocks
  137. Pingback: goyard handbags
  138. Pingback: jordan 6
  139. Pingback: ivory maxi vestito
  140. Pingback: jordan shoes
  141. Pingback: supreme
  142. Pingback: coach outlet
  143. Pingback: nike air force 1
  144. Pingback: supreme hoodie
  145. Pingback: nike shoes
  146. Pingback: kd 12
  147. Pingback: jordan shoes
  148. Pingback: cheap jordans
  149. Pingback: yeezy boost 350
  150. Pingback: birkin bag
  151. Pingback: curry 4 shoes
  152. Pingback: balenciaga shoes
  153. Pingback: bape hoodie
  154. Pingback: red bottom shoes
  155. Pingback: louboutin shoes
  156. Pingback: hermes belt
  157. Pingback: yeezy
  158. Pingback: hermes belt
  159. Pingback: nmd
  160. Pingback: moncler coat
  161. Pingback: longchamp outlet
  162. Pingback: kapesn铆 hodinky
  163. Pingback: pantofle s kaminky
  164. Pingback: llorens bambole
  165. Pingback: ama brand sneaker
  166. Pingback: adidas flux 35
  167. Pingback: michael kors scarf

Leave a Reply