Gitlab, Jenkins and puppet playing together

Jenkins in one of my favourite open source tools. It must also be one of the most flexible

You can set up Jenkins to monitor your Gitlab project for any changes. If Jenkins sees any commits to the master repo, Jenkins will execute a build script and run any customised scripts. Here is how to set this up.

• Set up a project in Gitlab. This will be your git repo.
• Create a your git repo using a git client on a linux box of your choice.
• Configure Jenkins job to poll the git repo and to run a build script if there are any changes.

Here is how to configure the Jenkins server to poll the git repo.

Install the gitlab plugin to Jenkins

https://wiki.jenkins-ci.org/display/JENKINS/Gitlab+Hook+Plugin

Set up a Jenkins job and specify the following under source code management

note: The repository URL can be obtained from gitlab as shown below

Setting up build triggers

Under build triggers, select the Poll SCM and set a Schedule. The schedule is in cron format and be set as desired. Here we have set Jenkins to poll the git repo every 5 minutes for changes.

Note: Once the job is setup, you can click on the

Here you will see the git polling working as shown below:

SSH keys and sudo access

We will need to set up ssh keys and sudo access for the jenkins_admin user which is used to log on to the puppet masters to sycnronise the manifests and modules. To set up the keys, run the following commands on the jenkins box.

1 – Log in as jenkins_admin@jenkins on the jenkins server

2 – Run the following command to drop into the jenkins user

sudo - s -H -u jenkins

3 – Run the following to set up the ssh key pair

$ cd ~/.ssh/
$ ssh-sopy-id -i id_rsa.pub jenkins_admin@puppet-server

note: # where puppet-server is the puppet master you need to sync the manifests on

4- Configure sudo on the puppet-master so Jenkins_admin is not prompted for the password. Note you can use a manifest to configure sudo via puppet.

on the puppet-server (as root)

sudo visudo

...

jenkins_admin ALL= NOPASSWD:/usr/bin/rsync

Make sure you do this process for each puppet-server you wish to configure sudo and ssh keys for

Configure the Jenkins build job

Configure the Jenkins build job as follows:

# Specifies a AD user who has sudo rights on the puppet server to run the rsync
commands without requiring a password

USER=jenkins_admin

# List of puppet masters that will receive manifests and modules

for HOST in 'puppet-server1t' 'puppet-server2' 'puppet-server3'

do

# rsync command to sync changes from gitlab repo

rsync -e "ssh -t -l jenkins_admin" --rsync-path='sudo rsync' -avz --stats
--progress --human-readable --exclude .git --exclude README $WORKSPACE/
$USER@$HOST:/etc/puppet/

done

This completes the setup.

GIT examples

I have been working on gitlab recently and needed to write down some git examples, so here is what I came up with:

1. To roll back files from your last commit

If the commit ID is ce4c1645cad94cc69221607201bd86c8f33b6cc0, run the following command

git reset --hard ce4c1645cad94cc69221607201bd86c8f33b6cc0

note: git reset without the –hard option resets the commit history but not the files. With the –hard option also files in working tree are reset

To then push these files back to the master branch

git commit --all
git push origin master --force

2. To recovery one file from a previous commit

To find out what files have been deleted from the previous commit, run

git log --diff-filter=D --summary

This should give you information about which commit the file was delete from. Once the commit ID is found, run the following command:

git checkout ce4c1645cad94cc69221607201bd86c8f33b6cc0 -- version_one_file_2

where ce4c1645cad94cc69221607201bd86c8f33b6cc0 is the commit ID and — version_one_file_2 is the file.

Then run the following commands:

git commit
git push

3 . Adding a FILE to a git repo

This is a very simple example but an important concept. Commands can vary depending on this example on what you are trying to do

create a new file

touch this_is_a_new_file

add the file to the git repo

git add this_is_a_new_file

this adds the file to the current working git directory

git commit this_is_a_new_file

this pushes the file to the master branchh

git push origin master

4. I deleted a file 3 commits ago. How do I recover that file?

How to reproduce the example

1. clone git repo

   see above :)

2. create a new file A

   touch A

3. Add the file to the git repo

   git add A

4. Commit and push the file A

   git commit A
   git push origin master

5. create a new file B and delete A

   touch B
   git add B
   git rm A

6. Commit and push the files

   git commit --all
   git push origin master

7. create a new file C

   touch C
   git add C

8. Commit and push the file C

   git commit --all
   git push origin master

Now try to find file A by running this command

git log --diff-filter=D --summary

it will show you what files have been deleted.

You need to checkout the previous branch; i.e. the commit before the deletion
You then need to make sure the file you are recoverying is the one you want
You then need to switch back to the master branch
You then need to add the file to the master branch
You then need to commit the recovered file
You then need to push everything to the repo

so..

git log --diff-filter=D --summary 
git checkout cdeb3d757f3adcc346da2ab171a825c113bdb50b~1 A

# note the ~1 rolls back to the previous commit. ~2 would go back 2 commits etc..

this just grabs that file in that commit and does not change branches

git branch

# check what branch you are on

git add A

# to add A back to the master branch

git commit A
git push origin master

5. Create a branch, adding file that does not conflict and adding the files to the master branch

Run git branch to show which branch you are using

git branch
* master

This shows you are using the master branch. To add a branch, run the following command

git branch new_branch
git branch

> * master 
  new_branch

The new_branch branch has been created. You have not checkout the new branch yet. First, the following command shows the files in the master branch.

git ls-files

 >INSTALL
  README
  config.xml

To switch to the new branch and to add a new file, run the following commands

git checkout new_branch

>Switched to branch 'new_branch'

git branch

> master
  * new_branch

notice the * is now next to the new_branch. This is to show which branch you are working on.

You can check the files are the same as the master branch by running

git ls-files

>  INSTALL
   README
   config.xml

Add a new file to the new_branch

touch file_added_to_branch
git add file_added_to_branch
git commit file_added_to_branch
git push origin new_branch

run the following to list the files in the new_branch

git ls-files

>  INSTALL
   README
   config.xml
   file_added_to_branch

switch to the master branch and list the git repo

git ls-files

>  INSTALL
   README
   config.xml

Notice that the file file_added_to_branch is not in the master repo. To add this file, you can merge the new_branch to the master repo by running the following command

 git merge new_branch

Updating 76a5cab..30c41cc Fast-forward 0 files changed, 0 insertions(+), 0 deletions(-)

create mode 100644 new_branch_file

Note: You have to be on the master branch to merge to the master. We changed to the master branch before running the above git merge command

push the files to the master branch on the git server

git push origin master

The following shows the branch files and how to delete any branches

git ls-files

>  INSTALL
   README
   config.xml
   new_branch_file

git branch

> * master
  new_branch

git branch -d new_branch

Deleted branch new_branch (was 30c41cc). 

git branch 

>  * master

Deleted the branch on the gitlab server

git push origin :new_branch
To git@puppet-gitlab:oliver.leach/puppet-cressex-lab.git
[deleted] new_branch