Jenkins is the way to manageable UI test automation
Test Automation Orchestration Engine
Better quality QA-as-a-Service is reliant on Jenkins and the Jenkins community.
Background: We built an infrastructure to run UI tests for both web and native applications that we run for several clients / projects. With Jenkins, we found a way to orchestrate the execution of hundreds of these tests on our test infrastructure, making sure that it is visual and manageable as well as making it easier to use a shared set of resources to run the test on, without them affecting each other.
Goals: Offering large-scale UI testing to multiple projects and clients worldwide.
Solution & Results: We needed a way to have executable tests in a easy-to-see-and-run process. Jobs are perfect for this, but we also needed a long running group of tests to be deployed at certain times. These might need to use shared resources, like containers that run selenium. Handling all of this could have been a nightmare, luckily with Jenkins it has been pretty smooth.
One of the things that helped us with this was the ease of configuring jobs to be run one after another, or one initial job triggering other jobs that can run in parallel. In short, being able to configure in detail when and how some of the tasks have to run.
Jenkins has been really easy to work with, there always seems to be a plugin or way to do what we needed. Plus the huge community means you're never alone. Lots of helpful Jenkins users around!
Another interesting point has been using groovy scripts to evaluate, for instance, if there are available machines running or if we need to start up some to run the job. So this integration to run groovy scripts has been very useful.
Some clients wanted to trigger a pipeline of jobs after a GitLab CI stage finished, so the Jenkins API has been really useful here. Enabling us to make this integration just the way we needed it, just with Curl.
The key capabilities we relied on were varied.
- Jenkins Groovy script support, which has been endlessly useful.
- Maven Plugin, to make building maven projects easier.
- Naginator Plugin, some jobs needed to be re-run in certain cases and this was pretty useful to easily achieve it.
- Pipeline plugin to build handy pipelines.
- Environment Injector plugin, to help us handle some job to job data.
- And of course, Git plugin, to easily configure the repository stuff.
We continue to be very happy with the results:
- Manage large quantities of tests with ease.
- Automate the execution of many jobs to minimize human interaction.
- Optimize the build to test time and reporting of results to stakeholders.