Table of Contents
1 Code samples for the Practical Devops book
These are the code samples for the book "Practical Devops" by Joakim Verona published by Packt Publishing.
The books home page: https://www.packtpub.com/networking-and-servers/practical-devops
In most cases you will need the book to make the most out of the exercises.
Please ensure that you have the latest version of these exercises before continuing.
To get the code samples from Github do:
git clone https://github.com/jave/practicaldevops.git
And to keep the samples updated,
cd practicaldevops
git pull https://github.com/jave/practicaldevops.git
1.1 ch3 How Architecture affects DevOps
1.1.1 Liquibase hello-world
cd ch3/liquibase-helloworld
mvn liquibase:update
1.1.2 Manual installation
dnf install postgresql
dnf install nginx
cd ch3/crm1
lein build
lein run
1.2 ch4 Everything is code
1.2.1 Docker intermission
These instructions are for Fedora, but they are similar for other distributions such as Ubuntu.
To make sure Docker is working properly, see the following documentation for Fedora.
https://docs.docker.com/v1.5/installation/fedora/
- For fedora 21 and later do:
dnf -y install docker
- docker-io was renamed to docker from Fedora 21, so use "docker-io" on older red hat derivates, "docker" on newer
- Use a sudo capable user to run docker commands, or the root user
- You can also add a docker group with rights to use the docker socket needed to communicate with the docker daemon.
This approach is described here https://docs.docker.com/v1.5/installation/fedora/
In summary:
$ sudo groupadd docker
$ sudo chown root:docker /var/run/docker.sock
$ sudo usermod -a -G docker $USERNAME
- You might need "setenforce 0" to start docker. The comand will disable selinux, which has security implications. Use this only on a test machine.
- To start and enable docker on reboot:
sudo systemctl start docker sudo systemctl enable docker
To verify that docker works:
sudo docker run -i -t fedora /bin/bash
For some exercises you need to have docker-compose installed first.
On Fedora 23 you can do:
dnf install docker-compose
In earlier versions you needed to download docker-compose manually.
1.2.2 setting up a basic git server
bare repo:
cd /opt/git mkdir project.git cd project.git git init --bare
- Now try cloning, making changes, and pushing to the server
1.2.3 Gerrit
Run a Gerrit container:
docker run -d -p 8080:8080 -p 29418:29418 openfrontier/gerrit
On the host machine you can now install the supporting git-review package:
sudo dnf install git-review
Rebase your commits on top of the commits in the remote repository:
git pull --rebase origin master
Interactively edit the history, possibly squashing commits together to make a more readable history:
git rebase -i origin/master
1.2.4 Gitlab
Now create a directory for gitlab, and fetch the compose file:
mkdir gitlab
cd gitlab
wget https://raw.githubusercontent.com/sameersbn/docker-gitlab/master/docker-compose.yml
Now start the gitlab stack.
docker-compose up
When the containers are up and running, access the web ui:
and enter the following credentials:
- username: root
- password: 5iveL!fe
1.3 ch5 Build the code
Create a "freestyle" class job in Jenkins that runs the "fortune" command.
First install Jenkins.
dnf install jenkins
Then follow the instruction in the book to configure the job.
1.3.1 Cheating with FPM
To install fpm:
yum install rubygems yum install ruby yum install ruby-devel gem install fpm
Package this shell script:
#!/bin/sh echo 'Hello World!' chmod a+x usr/local/bin/hello.sh fpm -s dir -t rpm -n hello-world -v 1 -C installdir usr rpm -qivp hello-world.rpm rpm -ivh hello-world.rpm
1.3.2 Build servers, Jenkins in particular
dnf install jenkins
systemctl start jenkins
1.4 ch6 Test the code
1.4.1 A Junit example
cd ch6/hello-junit
mvn install
1.4.2 Arquilian
There is an arquillian hello-world in the Arquillian documentation.
git clone https://github.com/aslakknutsen/arquillian-example-helloworld.git cd arquillian-example-helloworld mvn install
1.4.3 Automated acceptance testing
There are two implementations, one with annotations, and one with Lambda notation.
While the lambda notation is easier to read than the annotation syntax, cucumbers lambda notation is fairly new and can be problematic to get to work depending on your Java implementation.
To run the annotation based example:
cd ch6/hello-cucumber6 mvn clean test
To run the lambda based example:
cd ch6/hello-cucumber8 mvn clean test
1.4.4 A complete test automation scenario
- hello-selenium-world
Hello selenium world is a minimal selenium example that should open a firefox browser window and ask google 'hello world'. You should see a list of search matches for 'hello world'.
It is useful to check that this example runs before testing other examples. To run it:
cd ch6/hello-selenium mvn test - Running the usermanager example manually
You will need Leiningen, http://leiningen.org/ https://raw.githubusercontent.com/technomancy/leiningen/stable/bin/lein
ch6/usermanager lein run
- Running the automated test
autotest_v1/bin/autotest.sh
- Handling the tricky dependencies with Docker
docker run -d -p 4444:4444 --name selenium-hub selenium/hub docker run -d --link selenium-hub:hub selenium/node-firefox
1.5 ch7 Deploying the code
1.5.1 Executing code on the client
salt -E '.*' cmd.run 'ls -l'
1.5.2 Puppet master, Puppet agent
rfkrocktk/puppet is a convenient docker image for exploring puppet.
- https://hub.docker.com/r/rfkrocktk/puppet/ this is the agent
- https://hub.docker.com/r/rfkrocktk/puppetmaster/ this is the master
docker --name dockerduck --hostname dockerduck -e PUPPETMASTER_TCP_HOST=ultramaster.example.com \ -v /var/lib/docker/dockercontainer/puppet/ssl:/var/lib/puppet/ssl rfkrocktk/puppet
1.5.3 Ansible
FROM williamyeh/ansible:centos7
docker run -v `pwd`/ansible:/ansible -it <hash> bash cd /ansible ansible-playbook -i inventory playbook.yml --connection=local --sudo
A docker container which supports systemd:
FROM fedora RUN yum -y update; yum clean all RUN yum install ansible sudo RUN systemctl mask systemd-remount-fs.service dev-hugepages.mount \ sys-fs-fuse-connections.mount \ systemd-logind.service getty.target console-getty.service RUN cp /usr/lib/systemd/system/dbus.service /etc/systemd/system/;\ sed -i 's/OOMScoreAdjust=-900//' /etc/systemd/system/dbus.service VOLUME ["/sys/fs/cgroup", "/run", "/tmp"] ENV container=docker CMD ["/usr/sbin/init"]
To run the new container:
docker run -it --rm -v /sys/fs/cgroup:/sys/fs/cgroup:ro -v `pwd`/ansible:/ansible <hash>
Connect to the container:
docker exec -it <hash> bash
A slightly more advanced exercise:
--- - hosts: localhost vars: http_port: 80 max_clients: 200 remote_user: root tasks: - name: ensure apache is at the latest version yum: name=httpd state=latest - name: write the apache config file template: src=/srv/httpd.j2 dest=/etc/httpd.conf notify: - restart apache - name: ensure apache is running (and enable it at boot) service: name=httpd state=started enabled=yes handlers: - name: restart apache service: name=httpd state=restarted
1.5.4 Deploying with Chef
Start a clean container for the exercise:
docker run -it ubuntu bash
Install Chef in the container:
curl -L https://www.opscode.com/chef/install.sh | bash
Verify the chef-solo was installed:
chef-solo -v
Fetch and unpack a pre-rolled chef configuration:
curl -L http://github.com/opscode/chef-repo/tarball/master -o master.tgz tar -zxf master.tgz mv chef-repo* chef-repo rm master.tgz
Create a configuration file for chef:
mkdir .chef echo "cookbook_path [ '/root/chef-repo/cookbooks' ]" > .chef/knife.rb
Now create a template:
knife cookbook create phpapp
1.5.5 Deploying with Saltstack
Start a Saltstack container:
docker run -i -t --name=saltdocker_master_1 -h master -p 4505 -p 4506 \ -p 8080 -p 8081 -e SALT_NAME=master -e SALT_USE=master \ -v `pwd`/srv/salt:/srv/salt:rw jacksoncage/salt
Start a shell inside the Saltstack container:
docker exec -i -t saltdocker_master_1 bash
Salt state to install httpd:
top.sls: base: '*': - webserver webserver.sls: apache2: # ID declaration pkg: # state declaration - installed # function declaration
Run this command to ensure the desired state:
salt-call --local state.highstate -l debug
1.5.6 Vagrant
yum install 'vagrant*'
To use Vagrants Virtualbox driver, you need to set up Virtualbox according to your distribution.
Create a virtual machine with Vagrant from a recipy:
vagrant init hashicorp/precise32
Try starting the machine:
vagrant up
You can now ssh to the machine:
vagrant ssh
Add this to the Vagrant file:
Vagrant.configure("2") do |config| config.vm.box = "hashicorp/precise32" config.vm.provision :shell, path: "bootstrap.sh" end
And create a bootstrap.sh file that will install Apache httpd:
#!/usr/bin/env bash apt-get update apt-get install -y apache2
1.6 ch8 Monitoring the code
1.6.1 Nagios
Start a Nagios container:
docker run -e NAGIOSADMIN_USER=nagiosadmin -e NAGIOSAMDIN_PASS=nagios -p 80:30000 cpuguy83/nagios
Start a second container to monitor:
docker run -p 30001:80 nginx
A docker compose file for the scenario:
nagios:
image: mt-nagios
build:
- mt-nagios
ports:
- 80:30000
environment:
- NAGIOSADMIN_USER=nagiosadmin
- NAGIOSAMDIN_PASS=nagios
volumes:
./nagios:/etc/nagios
nginx:
image: nginx
Configuration files for the Nagios example:
define host {
name regular-host
use linux-server
register 0
max_check_attempts 5
}
define host{
use regular-host
host_name client1
address 192.168.200.15
contact_groups admins
notes test client1
}
hostgroups.cfg
define hostgroup {
hostgroup_name test-group
alias Test Servers
members client1
}
services.cfg
#+BEGIN_SRC sh
define service {
use generic-service
hostgroup_name test-group
service_description PING
check_command check_ping!200.0,20%!600.0,60%
}
An example mail configuration:
define contact{
contact_name matangle-admin
use generic-contact
alias Nagios Admin
email pd-admin@matangle.com
}
define contactgroup{
contactgroup_name admins
alias Nagios Administrators
members matange-admin
}
1.6.2 Munin
docker run -p 30005:80 lrivallain/munin:latest
Running commands in the munin container:
docker exec -it <hash> bash su - munin --shell=/bin/bash /usr/share/munin/munin-update
If you are having trouble running munin-update, try:
chown munin.munin /var/log/munin/munin-update.log
It may still take some time for the graphs to display.
This is the code for the munin plugin:
graph_title Load average graph_vlabel load load.label load
To emit data you simply print it to stdout.
printf "load.value " cut -d' ' -f2 /proc/loadavg
Here is an example script.
#!/bin/sh case $1 in config) cat <<'EOM' graph_title Load average graph_vlabel load load.label load EOM exit 0;; esac printf "load.value " cut -d' ' -f2 /proc/loadavg
1.6.3 Ganglia
To get help with the container:
docker run wookietreiber/ganglia --help
To run the Ganglia container:
docker run -p 30010:80 wookietreiber/ganglia
1.6.4 Graphite
Start Graphite:
docker run -it -p 30020:80 -p 2003:2003 sitespeedio/graphite
Try the following url: http://localhost:30020/
1.6.5 Log handling
Start Kibana and Elasticsearch:
docker run -d elasticsearch && docker run --link some-elasticsearch:elasticsearch -d kibana
1.7 ch9 Issue Tracking
1.7.1 Bugzilla
docker run -p 6050:80 dklawren/docker-bugzilla
1.7.2 Trac
docker run -d -p 6051:8080 barogi/trac:1.0.2
1.7.3 Redmine
docker run -d -p 6052:3000 redmine
1.7.4 The Gitlab issue tracker
Trying the Gitlab CLI:
GITLAB_API_PRIVATE_TOKEN=<token from your project> GITLAB_API_ENDPOINT=http://gitlab.matangle.com:50003/api/v3 gitlab help Issues
1.7.5 Jira
docker run -p 6053:8080 cptactionhank/atlassian-jira:latest
1.8 ch10 The Internet of Things and DevOps
1.8.1 NodeMCU
To get a newer firmware(please change the version to the latest available first):
wget https://github.com/nodemcu/nodemcu-firmware/releases/download/0.9.6-dev_20150704/nodemcu_integer_0.9.6-dev_20150704.bin
Get esptool:
git clone https://github.com/themadinventor/esptool.git
Install pyserial:
sudo dnf install pyserial
Burn the firmware:
sudo python ./esptool.py --port /dev/ttyUSB0 write_flash 0x00000 nodemcu_integer_0.9.6-dev_20150704.bin
You might need additional arguments:
sudo esptool.py --port=/dev/ttyUSB0 write_flash 0x0 nodemcu_integer_0.9.6-dev_20150704.bin -fs 32m -fm dio -ff 40m
Do some tests to see that the connection is working:
sudo ./esptool.py read_mac Connecting... MAC: 18:fe:34:00:d7:21 sudo ./esptool.py flash_id Connecting... Manufacturer: e0 Device: 4016
Try the LED:
gpio.write(0, gpio.LOW) -- turn led on
gpio.write(0, gpio.HIGH) -- turn led off
Blink the LED in a loop:
while 1 do -- loop forever gpio.write(0, gpio.HIGH) -- turn led off tmr.delay(1000000) -- wait one second gpio.write(0, gpio.LOW) -- turn led on tmr.delay(1000000) -- wait one second end
To connect to a wireless network.
wifi.setmode(wifi.STATION) wifi.sta.config("SSID","password")
To see the IP we got:
print(wifi.sta.getip())
Connecting to a web server:
conn=net.createConnection(net.TCP, false) conn:on("receive", function(conn, pl) print(pl) end) conn:connect(80,"121.41.33.127") conn:send("GET / HTTP/1.1\r\nHost: www.nodemcu.com\r\n" .."Connection: keep-alive\r\nAccept: */*\r\n\r\n")
Timer:
tmr.alarm(1, 1000, 1, function() print("hello world") end )
Stop the timer:
tmr.stop(1)