Building Quality Gates and Automated Decisions in CI/CD Pipelines. Lessons Learn... Dmitriy Gumeniuk

Building Quality Gates and Automated Decisions in CI/CD Pipelines. Lessons Learned with 200,000 Regressions. - Dmitriy Gumeniuk, EPAM Systems
Today’s modern development disciplines -- whether Agile, Continuous Integration (CI) or Continuous Delivery (CD) -- have completely transformed how teams develop and deliver applications faster. But in order to compete in today’s fast-paced digital economy, companies must also test faster and test continuously. Teams must make fast and reliable decisions by tracking test results of thousands tests for hundreds of runs triggered by code changes.
But what if we could leverage Machine Learning to do the analysis and help team make rule-based decisions using a concept named Quality Gates?
In this presentation, I will share learnings derived from regressions in 200,000 test cases, and how we use Machine Learning (ML) to triage failed reports and reduce team effort on results analysis. How we organized a Quality Gates and formed a rule-based engine to identify Go and No-Go decisions for a particular set. And how we automate release pipelines by use of meaningful feedback from the automated testing results.
You will learn how to build Quality Gates in ReportPortal, how to integrate feedback within CI/CD pipelines. How to utilize ML analysis in triaging of failed reports. And finally make a leap forward to Continuous Testing.
Outline/Structure
Learnings and pain points derived from 200,000 regression sets
Minimization of effort to triage failed reports
Extension of automated test cases with attribute, related to components, priority, severity, flows
Forming the dashboards for component health checks and environment checks
Building Quality Gates based on rules and test case attributes.
Organizing sequences of pipelines in Ci/CD tool based on automated QG decision
Outcome and time savings
Learning Outcome
You will learn how to :
Build Quality Gates in ReportPortal
Integrate test automation feedback within CI/CD pipelines
Extend automated test cases with attributes for detailed structuring
Utilize ML analysis for triaging of failed reports
Target Audience
QA Managers, QA Leads, Testing team leads, mid - senior test automation engineers
Links
Also known as Dzmitry Humianiuk.
Public Speaking experience:
Selenium Camp 2020, Ukraine, Kyiv talk "test-gap-analysis-and-minimization-of-regression-suites-via-test-to-code-mapping-with-drill4j"
DelEx Conf 2020, Belarus Talk: "Self-healing test automation with Healenium. Test Gap analysis and minimization of regression suite with drill4j"
SeleniumConf 2019, London, UK Talk: "Using ML to find value in your automated tests"
Automation in Action 2019, Ukraine, Kyiv Talk "App code coverage by automated tests - easy! We made test-to-code mapping open-sourced tool"
SeleniumCamp 2019, Ukraine, Kyiv Talk: "Building a self-service marketplace for Test Data"
DelEx Conf 2019, Belarus Talk: "Building self-managing Test Data marketplace"
top1@ TestingCup 2018, Poland Talk: "Machine Learning usage for test fail categorization"
top 5@ DelEx Conf 2018, Belarus Talk: "The applicability of machine learning and neural networks in test automation"
TestCon Moscow 2018,
SeleniumCamp 2018, Kiev, Ukraine
Belgrade TestConference 2017, Belgrade, Serbia, bg-testconferen...
Top 3@ TestConf 2017, Vilnius www.testcon.lt/
Top 5@ QAFest 2017, Kiev, Ukraine, qafest.com/
EPAM SEC2017, Budapest, Hungary events.epam.co...
BAQ Conference 2017, Lviv, Ukraine baq.dakiry.com.ua
top 10@ SeleniumCamp2017, Kiev seleniumcamp.com
top 6@ TestCon2017, Moscow testconf.ru
EPAM SEC 2016, 2014 (best speaker award)
SQA Days #17, #20 (Minsk, Belarus) sqadays.com
Selenium automates browsers. That's it! What you do with that power is entirely up to you. Primarily it is for automating web applications for testing purposes, but is certainly not limited to just that. Boring web-based administration tasks can (and should) also be automated as well. For more infomation check out: www.selenium.dev/