Nowadays, continuous integration is an important part of the agile software development life-cycle. There is a number of tools on the market: Atlassian Bamboo, Jenkins, Jetbrains TeamCity.
In my opinion, Jenkins has the most optimal product community and set of really useful plugins that suits most of your software projects: you can build software, deploy software, websites, portals to various places (e.g including AWS, DigitalOcean, bare metal servers) or to run unit tests. It can be integrated with communication tools of your choice, like Slack, HipChat or email.
If you haven’t had a chance to try Jenkins earlier, feel free to use the tutorial below to get started.
In order to install Jenkins, we will need: - Unix system. I would recommend a Debian-based machine, like Ubuntu server LTS - Java runtime environment installed. I usually use Java 8 - Get base Jenkins setup - Install necessary plugins - Put everything behind your web server.
The easiest way to install Java is through the apt-get
package
manager:
sudo apt-get install python-software-properties
sudo add-apt-repository ppa:webupd8team/java
sudo apt-get update
Once you added ppa
above, you can install java with the following
command:
sudo apt-get install oracle-java8-installer
You will need to execute a series of the following commands, namely: add the Jenkins signing key, register Jenkins apt sources, update package lists, and install Jenkins package.
wget -q -O - http://pkg.jenkins-ci.org/debian/jenkins-ci.org.key | sudo apt-key add -
sudo echo deb http://pkg.jenkins-ci.org/debian binary/ > /etc/apt/sources.list.d/jenkins.list
sudo apt-get update
sudo apt-get install jenkins
By default, it will install the base Jenkins setup, which is insecure.
You will need to go to the host where your Jenkins is installed, for
example: http://jenkins-host:8080/
.
Navigate to Manage Jenkins (on the left) and choose the “Configure
Global Security” item on the page loaded.
Now look below on the Matrix-based security (select it, if it is not selected previously), and make sure Anonymous only has the “Read” right under the View group.
Click Save at the bottom of the page. After the page reloads, you’ll see a login form, but simply ignore that and go to the home page (like, for example, http://jenkins-host:8080/). You’ll see this signup form, and the first signed up account will be the administrator.
Jenkins would not be so powerful without plugins. Usually, I install these plugins by default:
Bitbucket
: The BitBucket
plugin
is designed to offer integration between BitBucket and Jenkins.
BitBucket offers a Jenkins hook, but this one just triggers a build
for a specific job on commit, nothing more. The BitBucket plugin,
like the GitHub plugin, uses the POST hook payload to check which
job has to get triggered based on the changed repository/branch.bitbucket-pullrequest-builder
: This plugin builds pull requests
from Bitbucket.org. This is a must-have plugin if you perform QA
deploy for each submitted pull request.build-pipeline-plugin
: This plugin provides a Build Pipeline
View of upstream and downstream connected jobs that typically form
a build pipeline. In addition, it offers the ability to define
manual triggers for jobs that require intervention prior to
execution, e.g. an approval process outside of Jenkins. Provides
nice visualization of the paths & flows.copyartifact
: Adds a build step to copy artifacts from another
project. The plugin lets you specify which build to copy artifacts
from (e.g. the last successful/stable build, by build number, or by
a build parameter). You can also control the copying process by
filtering the files being copied, specifying a destination
directory within the target project, etc.credentials
: Adds a build step to copy artifacts from another
project. The plugin lets you specify which build to copy artifacts
from (e.g. the last successful/stable build, by build number, or by
a build parameter). You can also control the copying process by
filtering the files being copied, specifying a destination
directory within the target project, etc.delivery-pipeline-plugin
: Visualisation of Delivery/Build
Pipelines, renders pipelines based on upstream/downstream jobs.
When using Jenkins as a build server it is now possible with the
Delivery Pipeline Plugin to visualise one or more Delivery
Pipelines in the same view even in full screen.environment-script
: Environment Script
Plugin
allows you to have a script run after SCM checkout, before the
build. If the script fails (exit code isn’t zero), the build is
marked as failed. Any output on standard out is parsed as
environment variables that are applied to the build. It supports
“override syntax” to append paths to PATH-like variables.git
: Supports popular git version control system
ghprb
: This plugin builds pull requests in GitHub. It’s another
must-have plugin if your software development life cycle includes
deploying pull requests to PR environment to test.greenballs
: The funniest plugin - changes Jenkins to use green
balls instead of blue for successful builds.hipchat
: This plugin allows your team to setup build
notifications to be sent to HipChat rooms.To enable notifications,
add “HipChat Notifications” as a post-build step.junit
: Allows JUnit-format test results to be published. Note:
number of tools, including Karma, PhpUNIT & other tools allow to
publish test results in a JUnit format. Thus, this is a must-have
plugin for unit test flows.matrix-auth
: Offers matrix-based security authorization
strategies (global and per-project). This is quite handy if you
have shared build server across several teams.parameterized-trigger
: This plugin lets you trigger new builds
when your build has completed, with various ways of specifying
parameters for the new build.rebuild
: This plays nice with the parameterized-trigger plugin.
The plug-in allows the user to rebuild a parametrized build without
entering the parameters again.ssh
: You can use the SSH Plugin to run shell commands on a
remote machine via ssh.s3
: Allows uploading artifacts to S3 with multiple options.throttle-concurrents
: This plugin allows for throttling the
number of concurrent builds of a project running per node or
globally.Plugins are installed using Plugin manager on a Manage Jenkins Section.
Usually I hide Jenkins behind nginx. A typical configuration looks like the one below:
server {
listen 443 ssl;
server_name jenkins.vagrant.dev;
ssl_certificate /etc/nginx/jenkins_selfsigned.crt;
ssl_certificate_key /etc/nginx/jenkins_selfsigned.key;
location / {
proxy_pass http://127.0.0.1:8080;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_redirect off;
proxy_connect_timeout 150;
proxy_send_timeout 100;
proxy_read_timeout 100;
}
...
}
.gitmodules
git syntax,
specifies list of the dependencies that will be used by the playbook.
In particular, it includes ansible- by default developer_recipes
(repository with set of handy deployment recipes) and ansible role
called sa-box-bootstrap responsible for box securing steps
(assuming you plan to put Jenkins on remote hosts).[submodule "public/ansible_developer_recipes"]
path = public/ansible_developer_recipes
url = git@github.com:Voronenko/ansible-developer_recipes.git
[submodule "roles/sa-box-bootstrap"]
path = roles/sa-box-bootstrap
url = git@github.com:softasap/sa-box-bootstrap.git
[submodule "roles/sa-box-jenkins"]
path = roles/sa-box-jenkins
url = git@github.com:softasap/sa-box-jenkins.git
hosts - list here the initial box credentials that were provided to you for the server. Note: jenkins-bootstrap assumes you have the fresh box with root access only. If your box already secured, adjust credentials appropriately
[jenkins-bootstrap]
jenkins_bootstrap ansible_ssh_host=192.168.0.17 ansible_ssh_user=yourrootuser ansible_ssh_pass=yourpassword
[jenkins]
jenkins ansible_ssh_host=192.168.0.17 ansible_ssh_user=jenkins
jenkins_vars.yml - set here specific environment overrides, like your preferred deploy user name and keys.
jenkins_bootstrap.yml - First step - box securing. Creates jenkins user, and secures the box using sa-box-bootstrap role. See more details about the sa-box-bootstrap role In order, to override params for sa-box-bootstrap - pass the parameters like in the example below:
- hosts: all
vars_files:
- ./jenkins_vars.yml
roles:
- {
role: "sa-box-bootstrap",
root_dir: "{{playbook_dir}}/public/ansible_developer_recipes",
deploy_user: "{{jenkins_user}}",
deploy_user_keys: "{{jenkins_authorized_keys}}"
}
#!/bin/sh
ansible-playbook jenkins_bootstrap.yml --limit jenkins_bootstrap
ansible-playbook jenkins.yml --limit jenkins
jenkins_authorized_keys
(this is list of the keys, that allow
you to login to Jenkins box under Jenkins) - jenkins_domain
- your
agency domain - jenkins_host
- name of the Jenkins host (Site will
be bound to jenkins_host.jenkins_domain) - java_version
- your
Java choice (6,7,8 supported)jenkins_user: jenkins
jenkins_authorized_keys:
- "{{playbook_dir}}/components/files/ssh/vyacheslav.pub"
jenkins_domain: "vagrant.dev"
jenkins_host: "jenkins"
java_version: 8
jenkins_users
list of users with passwords to create. Admin and
deploy are required users.jenkins_users:
- {
name: "Admin",
password: "AAAdmin",
email: "no-reply@localhost"
}
- {
name: "deploy",
password: "DeDeDeDeploy",
email: "no-reply@localhost"
}
jenkins_plugins
Your choice of plugins to install. By default:jenkins_plugins:
- bitbucket # https://wiki.jenkins-ci.org/display/JENKINS/BitBucket+Plugin
- bitbucket-pullrequest-builder
- build-pipeline-plugin
- copyartifact # https://wiki.jenkins-ci.org/display/JENKINS/Copy+Artifact+Plugin
- credentials # https://wiki.jenkins-ci.org/display/JENKINS/Credentials+Plugin
- delivery-pipeline-plugin # https://wiki.jenkins-ci.org/display/JENKINS/Delivery+Pipeline+Plugin
- environment-script # https://wiki.jenkins-ci.org/display/JENKINS/Environment+Script+Plugin
- git
- ghprb # https://wiki.jenkins-ci.org/display/JENKINS/GitHub+pull+request+builder+plugin
- greenballs # https://wiki.jenkins-ci.org/display/JENKINS/Green+Balls
- hipchat # https://wiki.jenkins-ci.org/display/JENKINS/HipChat+Plugin
- junit # https://wiki.jenkins-ci.org/display/JENKINS/JUnit+Plugin
- matrix-auth # https://wiki.jenkins-ci.org/display/JENKINS/Matrix+Authorization+Strategy+Plugin
- matrix-project #https://wiki.jenkins-ci.org/display/JENKINS/Matrix+Project+Plugin
- parameterized-trigger #https://wiki.jenkins-ci.org/display/JENKINS/Parameterized+Trigger+Plugin
- rebuild # https://wiki.jenkins-ci.org/display/JENKINS/Rebuild+Plugin
- ssh
- s3 # https://wiki.jenkins-ci.org/display/JENKINS/S3+Plugin
- throttle-concurrents #https://wiki.jenkins-ci.org/display/JENKINS/Throttle+Concurrent+Builds+Plugin
You can download this template code from this repository. In order to use it - fork it, adjust parameters to your needs, and use.
Running is as simple as
./setup_jenkins.sh
Welcome to the world of continuous integration & deployment!