Jenkins is the way to build industry-leading log management

Building a Flexible and Modern DevOps Platform

Submitted By Jenkins User Donald Morton
This build and release manager set out to upgrade the company's Jenkins installation adding more flexibility and power to their DevOps platform.
Logo
Organization: Graylog, <https://www.graylog.org/>
Team Members: Donald Morton, Build and Release Engineer and Bernd Ahlers, Software Architect
Project Website: <https://github.com/Graylog2>
Programming Languages: Java, Node.js
Platform: : Docker, Kubernetes, Linux
Version Control System: GitHub
Build Tools: Maven
Community Support: Relied on Jenkins Users Google Group or IRC Chat, Spoke with colleagues and pee

Adding more flexibility and power into snapshot and release builds.

Background: Donald Morton was hired by Graylog in early 2020 and was given charge of their Jenkins installation. Graylog is a log management software company that centrally captures, stores, and enables real-time search and log analysis against machine data from any component in IT infrastructure and applications. The infrastructure was well designed, but it also hadn't been changed in about four years, so it was time for an upgrade.

Goals: More flexibility and power in our snapshot and release builds. Specifically, the goals were:

  1. Update Jenkins to the latest version so we can take advantage of the latest features.
  2. Use infrastructure-as-code as much as possible. Eliminate manual configuration. We should be able to destroy the VM and recreate it the same as it was before.
  3. Rewrite the jobs as declarative pipelines so that we can start to simplify and improve our complex builds.

Solution & Results: We started out writing a Terraform module to spin up infrastructure and an Ansible playbook for installing software on the machines. Jenkins itself runs off of a Docker image. The configuration of Jenkins is handled through the JCasC plugin.

Jenkins' flexibility is its real power. When you have complex needs like we do, you need something that can handle anything you can throw at it.
profile picture
Donald Morton, Build and Release Manager, Graylog

The freestyle jobs were rewritten in Groovy as declarative pipelines. We set up the Github Branch Source plugin, allowing Jenkins to auto-create jobs. We store the Groovy for the pipelines in git, allowing us to back out changes to jobs if something goes wrong. We did this because Declarative pipelines allow us to split our builds up into multiple stages and we have a shared library so that all our repos could take advantage of the same pipeline logic.

We just rebuild the Docker image, and it automatically updates Jenkins to the latest LTS and updates all plugins to the latest versions. It only takes a few minutes. The configuration of Jenkins itself is done through a single YAML file.

Capabilities we used:

  • Declarative Pipelines
  • Jenkins Shared Library
  • Jenkins Docker image
  • Jenkins Configuration-as-Code plugin
  • Github Branch Source plugin
  • Blue Ocean UI

Results so far:

  • Maintainability of job improved through infrastructure-as-code
  • Upgrade time went from never upgraded at all to upgrades within minutes
  • Observability of builds increased through Blue Ocean UI

Want the full story, visit Donald Morton'sGraylog Case Study.