Jenkins offers a simple way to set up a continuous integration or continuous delivery environment for almost any combination of languages and source code repositories using pipelines, as well as automating other routine development tasks. While Jenkins doesn’t eliminate the need to create scripts for individual steps, it does give you a faster and more robust way to integrate your entire chain of build, test, and deployment tools than you can easily build yourself.
“Don’t break the nightly build!” is a cardinal rule in software development shops that post a freshly built daily product version every morning for their testers. Before Jenkins, the best a developer could do to avoid breaking the nightly build was to build and test carefully and successfully on a local machine before committing the code. But that meant testing one’s changes in isolation, without everyone else’s daily commits. There was no firm guarantee that the nightly build would survive one’s commit.
Jenkins – originally Hudson – was a direct response to this limitation.
Hudson and Jenkins
In 2004, Kohsuke Kawaguchi was a Java developer at Sun. Kawaguchi became tired of breaking builds in his development work and wanted to find a way to know, before committing code to the repository, whether the code was going to work. So Kawaguchi built an automation server in and for Java to make that possible, called Hudson. Hudson became popular at Sun, and spread to other companies as open source.
Fast-forward to 2011, and a dispute between Oracle (which had acquired Sun) and the independent Hudson open source community led to a fork with a name change, Jenkins. Both forks continue to exist, although Jenkins is much more active. In 2014 Kawaguchi became CTO of CloudBees, which offers Jenkins-based continuous delivery products.
Jenkins automation
Today Jenkins is the leading open-source automation server with some 1,400 plugins to support the automation of all kinds of development tasks. The problem Kawaguchi was originally trying to solve, continuous integration and continuous delivery of Java code (i.e. building projects, running tests, doing static code analysis, and deploying) is only one of many processes that people automate with Jenkins. Those 1,400 plugins span five areas: platforms, UI, administration, source code management, and, most frequently, build management.
How Jenkins works
Jenkins is available as a Java 8 WAR archive and installer packages for the major operating systems, as a Homebrew package, as a Docker image, and as source code. The source code is mostly Java, with a few Groovy, Ruby, and Antlr files.
You can run the Jenkins WAR standalone or as a servlet in a Java application server such as Tomcat. In either case, it produces a web user interface and accepts calls to its REST API.
When you run Jenkins for the first time, it creates an administrative user with a long random password, which you can paste into its initial webpage to unlock the installation.
Jenkins plugins
Once installed, Jenkins allows you to either accept the default plugin list or choose your own plugins.
IDGOnce you have picked your initial set of plugins, click the Install button and Jenkins will add them.
IDGThe Jenkins main screen displays the current build queue and Executor status, and offers links to create new items (jobs), manage users, view build histories, manage Jenkins, look at your custom views, and manage your credentials.
IDGA new Jenkins item can be any of six types of job plus a folder for organizing items.
IDGThere are 18 things you can do from the Manage Jenkins page, including the option to open a command-line interface. At this point, however, we should look at pipelines, which are enhanced workflows that are typically defined by scripts.
IDGJenkins pipelines
Once you have Jenkins configured, it’s time to create some projects that Jenkins can build for you. While you can use the web UI to create scripts, the current best practice is to create a pipeline script, named Jenkinsfile, and check it into your repository. The screenshot below shows the configuration web form for a multibranch pipeline.
IDGAs you can see, branch sources for this kind of pipeline in my basic Jenkins installation can be Git or Subversion repositories, including GitHub. If you need other kinds of repositories or different online repository services, it’s just a matter of adding the appropriate plugins and rebooting Jenkins. I tried, but couldn’t think of a source code management system (SCM) that doesn’t already have a Jenkins plugin listed.
Jenkins pipelines can be declarative or scripted. A declarative pipeline, the simpler of the two, uses Groovy-compatible syntax—and if you want, you can start the file with #!groovy to point your code editor in the right direction. A declarative pipeline starts with a pipeline block, defines an agent, and defines stages that include executable steps, as in the three-stage example below.
pipeline {
agent any
stages {
stage(‘Build’) {
steps {
echo ‘Building..’
}
}
stage(‘Test’) {
steps {
echo ‘Testing..’
}
}
stage(‘Deploy’) {
steps {
echo ‘Deploying....’
}
}
}
}pipeline is the mandatory outer block to invoke the Jenkins pipeline plugin. agent defines where you want to run the pipeline. any says to use any available agent to run the pipeline or stage. A more specific agent might declare a container to use, for example:
agent {
docker {
image ‘maven:3-alpine’
label ‘my-defined-label’
args ‘-v /tmp:/tmp’
}
}stages contain a sequence of one or more stage directives. In the example above, the three stages are Build, Test, and Deploy.
steps do the actual work. In the example above the steps just printed messages. A more useful build step might look like the following:
pipeline {
agent any
stages {
stage(‘Build’) {
steps {
sh ‘make’
archiveArtifacts artifacts: ‘**/target/*.jar’, fingerprint: true
}
}
}
}Here we are invoking make from a shell, and then archiving any produced JAR files to the Jenkins archive.
The post section defines actions that will be run at the end of the pipeline run or stage. You can use a number of post-condition blocks within the post section: always, changed, failure, success, unstable, and aborted.
For example, the Jenkinsfile below always runs JUnit after the Test stage, but only sends an email if the pipeline fails.
pipeline {
agent any
stages {
stage(‘Test’) {
steps {
sh ‘make check’
}
}
}
post {
always {
junit ‘**/target/*.xml’
}
failure {
mail to: team@example.com, subject: ‘The Pipeline failed :(‘
}
}
}The declarative pipeline can express most of what you need to define pipelines, and is much easier to learn than the scripted pipeline syntax, which is a Groovy-based DSL. The scripted pipeline is in fact a full-blown programming environment.
For comparison, the following two Jenkinsfiles are completely equivalent.
Declarative pipeline
pipeline {
agent { docker ‘node:6.3’ }
stages {
stage(‘build’) {
steps {
sh ‘npm —version’
}
}
}
} Scripted pipeline
node(‘docker’) {
checkout scm
stage(‘Build’) {
docker.image(‘node:6.3’).inside {
sh ‘npm —version’
}
}
}Blue Ocean, the Jenkins GUI
If you’d like the latest and greatest Jenkins UI, you can use the Blue Ocean plugin, which provides a graphical user experience. You can add the Blue Ocean plugin to your existing Jenkins installation or run a Jenkins/Blue Ocean Docker container. With Blue Ocean installed, your Jenkins main menu will have an extra icon:
IDGYou can open Blue Ocean directly if you wish. It’s in the /blue folder on the Jenkins server. Pipeline creation in Blue Ocean is a bit more graphical than in plain Jenkins:
IDGJenkins Docker
As I mentioned earlier, Jenkins is also distributed as a Docker image. There isn’t much more to the process: Once you’ve picked the SCM type, you provide a URL and credentials, then create a pipeline from a single repository or scan all repositories in the organization. Every branch with a Jenkinsfile will get a pipeline.
Here I’m running a Blue Ocean Docker image, which came with a few more Git service plugins installed than the default list of SCM providers:
IDGOnce you have run some pipelines, the Blue Ocean plugin will display their status, as shown above. You can zoom in on an individual pipeline to see the stages and steps:
IDGYou can also zoom in on branches (top) and activities:
IDG—
IDGWhy use Jenkins?
The pipeline plugin we’ve been using supports a general continuous integration/continuous delivery (CICD) use case, which is probably the most common use for Jenkins. There are specialized considerations for some other use cases.
Java projects were the original raison d’être for Jenkins. We’ve already seen that Jenkins supports building with Maven; it also works with Ant, Gradle, JUnit, Nexus, and Artifactory.
Android runs a kind of Java, but introduces the issue of how to test on the wide range of Android devices. The Android emulator plugin allows you to build and test on as many emulated devices as you care to define. The Google Play Publisher plugin lets you send builds to an alpha channel in Google Play for release or further testing on actual devices.
I’ve shown examples where we specified a Docker container as the agent for a pipeline and where we ran Jenkins and Blue Ocean in a Docker container. Docker containers are very useful in a Jenkins environment for improving speed, scalability, and consistency.
There are two major use cases for Jenkins and GitHub. One is build integration, which can include a service hook to trigger Jenkins on every commit to your GitHub repository. The second is the use of GitHub authentication to control access to Jenkins via OAuth.
Jenkins supports many other languages besides Java. For C/C++, there are plugins to capture errors and warnings from the console, generate build scripts with CMake, run unit tests, and perform static code analysis. Jenkins has a number of integrations with PHP tools.
While Python code doesn’t need to be built (unless you’re using Cython, for instance, or creating a Python wheel for installation) it’s useful that Jenkins integrates with Python testing and reporting tools, such as Nose2 and Pytest, and code quality tools such as Pylint. Similarly, Jenkins integrates with Ruby tools such as Rake, Cucumber, Brakeman, and CI::Reporter.
Jenkins vs Bamboo
People often ask whether they’d be better off with Jenkins or Atlassian Bamboo for their CICD server. As a consultant, my stock answer is “It depends.”
To begin with, Jenkins is free open source, and Bamboo is commercial software. Jenkins has a bigger community and more plugins. On the other hand, Bamboo is well integrated out of the box with other Atlassian products such as Bitbucket, Jira, and Confluence.
It used to be that Bamboo was available as a cloud service as well as for on-premises deployments. As of January 2017, the Bamboo Cloud is no longer available; instead, Atlassian wants you to use Bitbucket Pipelines.
At this point, even though Bamboo Server is still available and can be hosted in a cloud-based container or VM, I suggest that people avoid it for new installations, on the grounds that it is becoming an orphan product. I do recommend considering Bitbucket Pipelines for CICD if you host your repositories on Bitbucket, and otherwise considering Jenkins.