Background: We built an infrastructure to run UI tests for both web and native applications that we run for several clients / projects. With Jenkins, we found a way to orchestrate the execution of hundreds of these tests on our test infrastructure, making sure that it is visual and manageable as well as making it easier to use a shared set of resources to run the test on, without them affecting each other.
Goals: Offering large-scale UI testing to multiple projects and clients worldwide.
Solution & Results: We needed a way to have executable tests in a easy-to-see-and-run process. Jobs are perfect for this, but we also needed a long running group of tests to be deployed at certain times. These might need to use shared resources, like containers that run selenium. Handling all of this could have been a nightmare, luckily with Jenkins it has been pretty smooth.
One of the things that helped us with this was the ease of configuring jobs to be run one after another, or one initial job triggering other jobs that can run in parallel. In short, being able to configure in detail when and how some of the tasks have to run.
Jenkins has been really easy to work with, there always seems to be a plugin or way to do what we needed. Plus the huge community means you're never alone. Lots of helpful Jenkins users around!
Another interesting point has been using groovy scripts to evaluate, for instance, if there are available machines running or if we need to start up some to run the job. So this integration to run groovy scripts has been very useful.
Some clients wanted to trigger a pipeline of jobs after a GitLab CI stage finished, so the Jenkins API has been really useful here. Enabling us to make this integration just the way we needed it, just with Curl.
The key capabilities we relied on were varied.
We continue to be very happy with the results: