The CICD Dilemma

Agiletestware works with a variety of engineering organization all over the world. It is always enlightening to see how different organization implement quality engineering discipline and try to achieve CICD (continuous integration and continuous delivery). Our experience shows that CICD is a long journey and requires a great amount of experimentation, discipline, and custom integrations with a variety of tools and frameworks. There is no silver bullet that will get you to the promised land 😢 CICD takes time, patience, and a good vision 😇

Many of our customers use a wide variety of testing frameworks, tools, CI systems, ALM solution, bug trackers, requirement management systems, etc. All of them want to achieve CICD but integrating all these technologies is a very challenging task when you need to do your day job and ship features to customers.

In these series of blog posts, we will go over how many of our customers have come one step closer to achieving CICD. The blog posts will explain how a variety of testing frameworks and tools and have been integrated with HP ALM (our customer’s ALM software for tracking projects) using Agiletestware’s Bumblebeeproduct.

The typical technology landscape at most large companies

Before we explain the solution, let us explain the typical setup found in many of our customers. Hopefully, many of you will relate to this situation.

CI Applications – Jenkins, Bamboo, and TeamCity

CI system are at the center of achieving CICD. The three biggest and most mature CI applications are JenkinsAtlassian Bamboo, and Jetbrain’s TeamCity. CI systems are used to build software, run tests, and do a variety of other tasks. In mature engineering organizations, CI systems are used to deploy software to achieve CD. All the information about these various tasks (build, test results, etc) are contained within the CI application.

What is HP ALM?

HP ALM is the leading solution in the industry for tracking complex software project. HP ALM is used for bug tracking, requirements management, test case management, and much more. HP ALM is a commercial product and integrates with various other tools from HP tools for testing web, mobile, and various other software products.

Web Testing and Selenium WebDriver

Selenium Webdriver is an open source tool for automating browsers. Primarily, it is for automating web applications for testing purposes. Selenium has the support of some of the largest browser vendors who have taken (or are taking) steps to make Selenium a native part of their browser. It is also the core technology in countless other browser automation tools, APIs and frameworks. If you are not using Selenium for testing web applications, you should re-think your testing tool strategy and take a closer look at Selenium WebDriver 😀

Mobile Testing and Appium

Appium is an open source test automation framework used for testing native, hybrid and mobile web apps. Appium aims to automate ios, Android, and Windows mobile app from any modern programming language and any test framework. Appium is supported by SauceLabs (leading provider of web and mobile test cloud). Appium is most widely used mobile testing framework and is free.

JAVA Test Frameworks – testNG and JUnit

Testing tools like Selenium and Appium require some test framework. In the JAVA world, two of the most widely used test frameworks are JUnit and testNG.

JUnit: JUnit is a simple framework to write repeatable tests. It is an instance of the xUnit architecture for unit testing frameworks.

testNG: TestNG is a testing framework inspired from JUnit and NUnit but introducing some new functionalities that make it more powerful and easier to use.

All sounds simple. So what is the problem?

If you are a HP ALM shop and using HP tools (UFT, LeanFT, QTP, etc) for testing all your applications, then there are no issues and life should be good. Every software development and testing activity should nicely be reflected in HP ALM.

However, the reality is that most organizations are rarely a homogeneous HP shop. Some teams may be using HP Tools (UFT, LeanFT, QTP), mobile testing team may be using Appium or other similar open source tools, browser application testing teams may be using Selenium WebDriver and cloud browser solutions like SauceLabs and BrowserStack, API testing team may be using using ReadyAPI, and unit test is hopefully done via some x-unit framework like Junit or testNG and most of these activities are controlled via Jenkins/Bamboo/TeamCity.

The end result is that you end up with islands of information within your organization. Some information is in HP ALM and some is in Jenkins/Bamboo/Teamcity and it is impossible to get a unified view of the project. As a consequence, engineering management often makes an uninformed decision because they don’t know how to collect all the data. There are just too much human element, too many reports to prepare, and it is just a pain and no one wants to do this.

How can Agiletestware help?

Agiletestware has been helping engineering organizations achieve CICD through many of our tools. We can’t solve all your problems but we may be able to solve many of them to get you closer to CICD. HP ALM is a significant investment and you should get the most out of your investment. We believe HP ALM is still the best ALM solution in the market and has the most mature bug tracking, agile project management, requirements management, and test case management modules in a single solution.

Agiletestware’s Bumblebee product can help integrate CI applications like Jenkins/Bamboo/Teamcity, JAVA tests frameworks like testNG/JUnit, and web and mobile testing tools like Selenium WebDriver / Appium with HP ALM with little to no coding. Bumblebee simply intercepts activities within your existing testing tools and CI frameworks and exports it to HP ALM in real-time.


I am the founder of Agiletestware. I have been working as a Quality Engineering engineer, manager, architect, and consultant for about 10+ years in Los Angeles, and San Francisco. I have worked with a variety of test frameworks, test infrastructure automation strategies and tools, and various ALM and test case management applications. If you need advice on how to make your teams and processes efficient, transparent, and how best to integrate testing into your CICD strategy, shoot me an email and I would love to chat.

1,253 thoughts on “The CICD Dilemma”

  1. Pingback: Nike Element 87
  2. Pingback: jordan retro 12
  3. Pingback: nike air max
  4. Pingback: nike air max 90
  5. Pingback: Basketball Jersey
  6. Pingback: air jordans
  7. Pingback: jordan shoes
  8. Pingback: adidas zx flux
  9. Pingback: Balenciaga
  10. Pingback: moncler
  11. Pingback: Nike Outlet Online
  12. Pingback: Nike Outlet Store
  13. Pingback: golden goose
  14. Pingback: jordan shoes
  15. Pingback: yeezy
  16. Pingback: kyrie 3
  17. Pingback: adidas yeezy
  18. Pingback: nike air max
  19. Pingback: Yeezy 350
  20. Pingback: Yeezy 350
  21. Pingback: nike flyknit
  22. Pingback: yeezy 700
  23. Pingback: Nike Outlet Store
  24. Pingback: jordans
  25. Pingback: nike cortez
  26. Pingback: yeezy boost
  27. Pingback: Yeezy
  28. Pingback: Yeezys
  29. Pingback: off white hoodie
  30. Pingback: Adidas Yeezy
  31. Pingback: Yeezy
  32. Pingback: air max 270
  33. Pingback: adidas superstar
  34. Pingback: fila shoes
  35. Pingback: yeezy boost
  36. Pingback: air max
  37. Pingback: yeezy boost 350
  38. Pingback: jordan 12
  39. Pingback: golden goose
  40. Pingback: kd shoes
  41. Pingback: kd shoes
  42. Pingback: nike shox for men
  43. Pingback: supreme new york
  44. Pingback: kd shoes
  45. Pingback: nba jerseys
  46. Pingback: polo ralph lauren
  47. Pingback: nike air max
  48. Pingback: adidas yeezy
  49. Pingback: air max 97
  50. Pingback: adidas nmd
  51. Pingback: air max 97
  52. Pingback: timberland outlet
  53. Pingback: nike vapormax
  54. Pingback: yeezy boost
  55. Pingback: longchamp handbags
  56. Pingback: kyrie shoes
  57. Pingback: jimmy choo
  58. Pingback: nike air max 270
  59. Pingback: kobe sneakers
  60. Pingback: nike air max 2017
  61. Pingback: nike air max95
  62. Pingback: jordan shoes
  63. Pingback: hermes belt
  64. Pingback: yeezys
  65. Pingback: kyrie 5 shoes
  66. Pingback: balenciaga speed
  67. Pingback: ferragamo belt
  68. Pingback: nike air vapormax
  69. Pingback: yeezy boost 350 v2
  70. Pingback: yeezys
  71. Pingback: air jordan
  72. Pingback: jordan shoes
  73. Pingback: golden goose
  74. Pingback: coach outlet store
  75. Pingback: louboutin outlet
  76. Pingback: coach outlet
  77. Pingback: nike shoes
  78. Pingback: cheap jordans
  79. Pingback: moncler outlet
  80. Pingback: yeezy boost 350 v2
  81. Pingback: nike 270
  82. Pingback: nike air max
  83. Pingback: air jordans
  84. Pingback: mbt
  85. Pingback: lebron 15
  86. Pingback: supreme shirt
  87. Pingback: kyrie 5
  88. Pingback: hermes
  89. Pingback: off white clothing
  90. Pingback: MLB Jerseys
  91. Pingback: lebron 17
  92. Pingback: longchamp handbags
  93. Pingback: hermes birkin
  94. Pingback: rock and blue kort
  95. Pingback: air jordan
  96. Pingback: louboutin
  97. Pingback: jordan retro
  98. Pingback: converse outlet
  99. Pingback: coach outlet
  100. Pingback: dainese pro armor
  101. Pingback: vans shoes
  102. Pingback: adidas tubular x
  103. Pingback: yeezy
  104. Pingback: yeezy boost 350
  105. Pingback: curry shoes
  106. Pingback: supreme clothing
  107. Pingback: kd shoes
  108. Pingback: chinese 12s
  109. Pingback: grote maten h m
  110. Pingback: kyrie 6
  111. Pingback: yeezy boost 350
  112. Pingback: koti wale lehenga
  113. Pingback: supreme outlet
  114. Pingback: sautoir pas cher
  115. Pingback: chaussure clou
  116. Pingback: kd 12
  117. Pingback: golden goose
  118. Pingback: nike shox white
  119. Pingback: chrome hearts
  120. Pingback: hermes birkin
  121. Pingback: zulily harem pants
  122. Pingback: irving shoes
  123. Pingback: jordan retro
  124. Pingback: nike air max
  125. Pingback: pure boost red
  126. Pingback: huawei honor 8 pro
  127. Pingback: casual ecco o2 178
  128. Pingback: goyard bags
  129. Pingback: longchamp handbags
  130. Pingback: kyrie irving shoes
  131. Pingback: nike air max 2018
  132. Pingback: nike air max
  133. Pingback: convese outlet
  134. Pingback: nike air max 90
  135. Pingback: off white
  136. Pingback: high neck frocks
  137. Pingback: goyard handbags
  138. Pingback: jordan 6
  139. Pingback: ivory maxi vestito
  140. Pingback: jordan shoes
  141. Pingback: supreme
  142. Pingback: coach outlet
  143. Pingback: nike air force 1
  144. Pingback: supreme hoodie
  145. Pingback: nike shoes
  146. Pingback: kd 12
  147. Pingback: jordan shoes
  148. Pingback: cheap jordans
  149. Pingback: yeezy boost 350
  150. Pingback: birkin bag
  151. Pingback: curry 4 shoes
  152. Pingback: balenciaga shoes
  153. Pingback: bape hoodie
  154. Pingback: red bottom shoes
  155. Pingback: louboutin shoes
  156. Pingback: hermes belt
  157. Pingback: yeezy
  158. Pingback: hermes belt
  159. Pingback: nmd
  160. Pingback: moncler coat
  161. Pingback: longchamp outlet
  162. Pingback: kapesn铆 hodinky
  163. Pingback: pantofle s kaminky
  164. Pingback: llorens bambole
  165. Pingback: ama brand sneaker
  166. Pingback: adidas flux 35
  167. Pingback: michael kors scarf
  168. Pingback: sqsmgs
  169. Pingback: feqahat
  170. Pingback: duetosa
  171. Pingback: casque audio being
  172. Pingback: urtubei
  173. Pingback: flaflooga
  174. Pingback: scribchats
  175. Pingback: radiusm
  176. Pingback: tiawards
  177. Pingback: moviezpk
  178. Pingback: brucelaval
  179. Pingback: madisllc
  180. Pingback: sorrowline
  181. Pingback: fbcont
  182. Pingback: gant ski leki
  183. Pingback: solerbe
  184. Pingback: bowiet
  185. Pingback: detikwin
  186. Pingback: nissancl
  187. Pingback: turiviaje
  188. Pingback: bsfturf
  189. Pingback: fsweny
  190. Pingback: ediquid
  191. Pingback: hokimerah
  192. Pingback: bermaee
  193. Pingback: bgbonus
  194. Pingback: mcsnetzone
  195. Pingback: evapter
  196. Pingback: laviafla
  197. Pingback: sumafood
  198. Pingback: tlazohtla
  199. Pingback: kiuhoki
  200. Pingback: hilbahis
  201. Pingback: deisida
  202. Pingback: desporto loja
  203. Pingback: coreydc
  204. Pingback: rtnplus
  205. Pingback: czapki kamiland
  206. Pingback: depoindo
  207. Pingback: boncareusa
  208. Pingback: gliclub
  209. Pingback: champion tshirt
  210. Pingback: sitesvtt
  211. Pingback: murziez
  212. Pingback: zonagg
  213. Pingback: ferrisstore
  214. Pingback: trinauinc
  215. Pingback: simetube
  216. Pingback: flindey
  217. Pingback: djenga
  218. Pingback: romanoilo
  219. Pingback: pacombat
  220. Pingback: norcalsba
  221. Pingback: mpishere
  222. Pingback: sanbilizi
  223. Pingback: ufcfree
  224. Pingback: ecmsracin
  225. Pingback: fanilista
  226. Pingback: cowesfc
  227. Pingback: bmrocks
  228. Pingback: caribhash
  229. Pingback: glogon
  230. Pingback: tatilcitur
  231. Pingback: chicloca
  232. Pingback: pravolib
  233. Pingback: Yeezy 380
  234. Pingback: Nike Shoes
  235. Pingback: dominiclb
  236. Pingback: directscl
  237. Pingback: cgahio
  238. Pingback: farsmizban
  239. Pingback: swdhaka
  240. Pingback: cpacurrie
  241. Pingback: monobrot
  242. Pingback: uzmanpet
  243. Pingback: taylorsifts
  244. Pingback: mrchiizu
  245. Pingback: tomecko
  246. Pingback: nasirahoori
  247. Pingback: cellardoves
  248. Pingback: virusloft
  249. Pingback: hotlimon
  250. Pingback: cazoletos
  251. Pingback: natiasud
  252. Pingback: eshwrites
  253. Pingback: dkebabs
  254. Pingback: nlmluo
  255. Pingback: enkuri
  256. Pingback: rsbengg
  257. Pingback: darzarzis
  258. Pingback: hopcoz
  259. Pingback: brandiali
  260. Pingback: maeblooms
  261. Pingback: nlmczech
  262. Pingback: simtivo
  263. Pingback: limoind
  264. Pingback: kreshme
  265. Pingback: hariharagro
  266. Pingback: monsterants
  267. Pingback: battlebrik
  268. Pingback: bordalba
  269. Pingback: hultcos
  270. Pingback: sksiberians
  271. Pingback: soldes ugg femme
  272. Pingback: croncas
  273. Pingback: kaewwern
  274. Pingback: kamillesk
  275. Pingback: vesissb
  276. Pingback: rinajordi
  277. Pingback: nsihorwitz
  278. Pingback: hondaview
  279. Pingback: songkolo
  280. Pingback: tbconnor
  281. Pingback: scriptmask
  282. Pingback: kmcases
  283. Pingback: nlmkorea
  284. Pingback: be谩ll铆t谩s
  285. Pingback: moda 2000 near me
  286. Pingback: 670 nb
  287. Pingback: sasosauces
  288. Pingback: flaresco
  289. Pingback: nautsat
  290. Pingback: faithask
  291. Pingback: sehrinet
  292. Pingback: soanuncios
  293. Pingback: stcglaw
  294. Pingback: baotinhoc
  295. Pingback: muroumido
  296. Pingback: wizdomav
  297. Pingback: webmanisa
  298. Pingback: sericorse
  299. Pingback: porta sella
  300. Pingback: ica erbjudande
  301. Pingback: pochette tortora
  302. Pingback: felpe online
  303. Pingback: kurs jogi warszawa
  304. Pingback: straight and curl
  305. Pingback: bezvabeh
  306. Pingback: baankru
  307. Pingback: tostseks
  308. Pingback: bearbu
  309. Pingback: boncarry
  310. Pingback: martinraes
  311. Pingback: swccpa
  312. Pingback: elhosh
  313. Pingback: dorisburke
  314. Pingback: ciruscg
  315. Pingback: envifoam
  316. Pingback: salesviet
  317. Pingback: lussaro
  318. Pingback: twaitor
  319. Pingback: tetlanews
  320. Pingback: kabreeta
  321. Pingback: ductails
  322. Pingback: fast hair brush
  323. Pingback: caftravel
  324. Pingback: mazdakism
  325. Pingback: theawco
  326. Pingback: ersions
  327. Pingback: tufcards
  328. Pingback: hanoibnb
  329. Pingback: utahcarzz
  330. Pingback: nusmata
  331. Pingback: balzacmall
  332. Pingback: lastpil
  333. Pingback: daarolelm
  334. Pingback: florisha
  335. Pingback: ratutoto
  336. Pingback: bestik k酶b
  337. Pingback: wernerskl
  338. Pingback: stcoso
  339. Pingback: thinkeet
  340. Pingback: soundohio
  341. Pingback: mersinmasoz
  342. Pingback: siselup
  343. Pingback: suprema 75ah 750a
  344. Pingback: profoto b10 fimeko
  345. Pingback: cariobat
  346. Pingback: suncalla
  347. Pingback: visitfame
  348. Pingback: norcodrene
  349. Pingback: celiatys
  350. Pingback: luisagaton
  351. Pingback: vinosvin
  352. Pingback: saswimsuit
  353. Pingback: ehsonh
  354. Pingback: nccolc
  355. Pingback: vnuwill
  356. Pingback: zlatan em tr枚ja
  357. Pingback: ewgaar
  358. Pingback: bundy pre deti
  359. Pingback: seiko prezzi
  360. Pingback: lenzayas
  361. Pingback: boris godal
  362. Pingback: limooh
  363. Pingback: fitraves
  364. Pingback: svbola
  365. Pingback: beanninc
  366. Pingback: matuteo
  367. Pingback: natovka
  368. Pingback: smiltur
  369. Pingback: vizcake
  370. Pingback: work boots price
  371. Pingback: 34 bra size
  372. Pingback: komplet bryggers
  373. Pingback: kapsar do koupelny
  374. Pingback: veste enfants
  375. Pingback: rokas trimmeris
  376. Pingback: vekiltv
  377. Pingback: mundeinc
  378. Pingback: sat na dodir
  379. Pingback: pulseras en piel
  380. Pingback: disney moana dress
  381. Pingback: mode comma 2019
  382. Pingback: open sesame street
  383. Pingback: meble na ogr贸d
  384. Pingback: maison canap茅
  385. Pingback: ezperdeim
  386. Pingback: betasahm
  387. Pingback: turkrotts
  388. Pingback: zombess
  389. Pingback: hochede
  390. Pingback: nivacharter
  391. Pingback: ureditor
  392. Pingback: bilabeela
  393. Pingback: bafuinn
  394. Pingback: hscuae
  395. Pingback: buildemo
  396. Pingback: sulesalon
  397. Pingback: qmcdot
  398. Pingback: madspo
  399. Pingback: meduzapw
  400. Pingback: brennanoil
  401. Pingback: loserdad
  402. Pingback: zeubros
  403. Pingback: wtmiradio
  404. Pingback: klopbali
  405. Pingback: viasalta
  406. Pingback: limademi
  407. Pingback: baileysms
  408. Pingback: dictatorcms
  409. Pingback: mobehowto
  410. Pingback: elainewlane
  411. Pingback: lovemailru
  412. Pingback: valizze
  413. Pingback: selltcr
  414. Pingback: miuidy
  415. Pingback: napelinos
  416. Pingback: sillypea
  417. Pingback: hurfar
  418. Pingback: flamepm
  419. Pingback: mealspeal
  420. Pingback: lewilou
  421. Pingback: tmfeatures
  422. Pingback: sertce
  423. Pingback: mimirobic
  424. Pingback: smilepronto
  425. Pingback: tazenin
  426. Pingback: rmeisner
  427. Pingback: vindefruit
  428. Pingback: pulsastar
  429. Pingback: braporno
  430. Pingback: sibelicious
  431. Pingback: eleandrah
  432. Pingback: larmovie
  433. Pingback: flowkati
  434. Pingback: viettoi
  435. Pingback: snapcre
  436. Pingback: gparmc
  437. Pingback: lroseinc
  438. Pingback: susanavello
  439. Pingback: langlerol
  440. Pingback: evrofinans
  441. Pingback: borinsel
  442. Pingback: seesjobs
  443. Pingback: sockcop
  444. Pingback: horntees
  445. Pingback: koduvein
  446. Pingback: opensats
  447. Pingback: csmfirm
  448. Pingback: discnerd
  449. Pingback: calusseau
  450. Pingback: urbanhit
  451. Pingback: cotmanip
  452. Pingback: cowsunite
  453. Pingback: conzones
  454. Pingback: cullmannaz
  455. Pingback: civasclinic
  456. Pingback: daaelite
  457. Pingback: entkalker
  458. Pingback: codingmedia
  459. Pingback: haloresi
  460. Pingback: eastoverfc
  461. Pingback: sarangbang
  462. Pingback: torrentmr
  463. Pingback: eduworlds
  464. Pingback: mandrait
  465. Pingback: boredgourd
  466. Pingback: sakshotel
  467. Pingback: mdroster
  468. Pingback: visvita
  469. Pingback: motalefeh
  470. Pingback: eduzhituo
  471. Pingback: ecosolder
  472. Pingback: neteuros
  473. Pingback: meetpukie
  474. Pingback: gizmoart
  475. Pingback: freimap
  476. Pingback: marganza
  477. Pingback: dmmllaw
  478. Pingback: holotc
  479. Pingback: forascie
  480. Pingback: koiarise
  481. Pingback: sfllaw
  482. Pingback: cuenco
  483. Pingback: itcolate
  484. Pingback: viaishere
  485. Pingback: glennraps
  486. Pingback: leomedicus
  487. Pingback: cosuisse
  488. Pingback: hurlabs
  489. Pingback: netlinebr
  490. Pingback: siames
  491. Pingback: radiocci
  492. Pingback: mimsstores
  493. Pingback: sueflowers
  494. Pingback: napscorp
  495. Pingback: lowlande
  496. Pingback: fromeaparty
  497. Pingback: wilbanksel
  498. Pingback: samisuka
  499. Pingback: dfmfit
  500. Pingback: leanlaw
  501. Pingback: nashekafe
  502. Pingback: alberco
  503. Pingback: buzzages
  504. Pingback: fullonrad
  505. Pingback: esitelecom
  506. Pingback: haresmoor
  507. Pingback: marineclean
  508. Pingback: proelbra
  509. Pingback: mersinulus
  510. Pingback: Yeezy 350
  511. Pingback: Yeezy 350
  512. Pingback: UGG Outlet
  513. Pingback: Timberland UK
  514. Pingback: Nike Shoes
  515. Pingback: Adidas Yeezy
  516. Pingback: Yeezy Shoes
  517. Pingback: Adidas Yeezy
  518. Pingback: Adidas yeezy
  519. Pingback: Timberland

Leave a Reply