cloudstack

Cloudstack deploying cloudstack

Image

I was talking at the Cloudstack European Meet up the other day about getting Cloudstack to deploy Cloudstack.

Well its more than that – I was talking about the concepts rather than what I was doing, as really, Cloudstack deploying Cloudstack is a bit of a gimick.

Here is my github link to the project
https://github.com/oliverleach/cloudstack-puppet

However, I thought I would include some scribble on how the project works in case anyone is interested:

Cloudstack 4.0.1 deploy

This project enables a running Cloudstack instance 3.x and above to deploy Cloudstack 4.0.1 instances. This could be to demo Cloudstack 4.0.1 or used for testing purposes, by easily spinning up Cloudstack 4.0.1 instances. It could also be used to provision private clouds using Cloudstack.

How to use this project

The python script and puppet manifests in this repo deploys cloudstack 4.0.1 using Puppet and the Cloudstack API. For this to work, you need to use the python scripts and 2 puppet templates. One template is the puppet-master, and one template is the puppet-agent.

You need to configure the python parameters in the puppet-deploy.py script as per your environment. This includes:

1 – your API key
2 – your secret key
3 – your service offering
4 – your Cloudstack zone ID
5 – your network ID of your user account
6 – The Cloudstack provisioning host IP and port
7 – The Cloudstack protocol used (http or https)

You will also need your 2 template IDs for the puppet master and puppet agent

These templates can be downloaded using the following link:

Puppet Agent

https://dl.dropbox.com/u/79905244/puppet-agent.7z

Puppet Master

https://dl.dropbox.com/u/79905244/puppet-master.7z

Both templates are running CentOS 6.4 64-bit. The puppet master is configured as a puppet-ca, a puppet server and a puppet agent. The puppet agent has the puppet agent installed and runs a userdata.rb script on startup. See the rc.local file for further information.

Once the templates are unzipped and uploaded in to your provisioning Cloudstack instance, update the puppet-deploy.py script with the template IDs.

To run the python scripts, use an interpreter like idle, open and run the puppet-deploy.py file, or simply run from the command line where an instance of python is installed; e.g. python puppet-deploy.py. This has been tested against python 2.7.3.

The git repo holds the puppet manifests which are included in the puppet-master template.
They are just for reference.

This project is just for fun, could be better written and I hold no responsibility if you use the project. You are allowed to do so.

Advertisements

Puppet manifests for cloudstack and co.

Here are some of my puppet manifests for Cloudstack and co. You can also find more information here:

https://github.com/oliverleach/cloudstack-puppet

I also recently did a presentation at the Cloudstack European meetup in London on the 11th April about deploying applications using puppet and Cloudstack. The talk was well received so I thought I would jot down the manifests that I wrote as part of the talk.

Here is the Cloudstack manifest:


# Class cloudstack
#
# This class installs cloudstack
#
# Author: Oliver Leach
# Date : 18th March 2013
#
# Actions:
# Installs the cloudstakc package
#
# This includes, the cloudstack-client, mysql-server and any install commands
#
# Currently the clodustack class is monlithic for demo purposes.
#
# Requires:
# Centos - other operating systems are not supported by this manifest
#
# Sample Usage:
# include cloudstack
#

class cloudstack ($mysql_password) {

case $::operatingsystem {

'centos': {
$supported = true
}
}

if ($supported == true) {

yumrepo { 'cloudstack':
baseurl  => "http://cloudstack.apt-get.eu/rhel/4.0/",
descr    => "Cloudstack 4.0.1 yum repo",
enabled  => 1,
gpgcheck => 0,
}

package { "cloud-client":
ensure   => "installed",
require  => Yumrepo["cloudstack"],
}

exec { "cloud-setup" :
command  => "/usr/bin/cloud-setup-databases cloud:cloud@localhost --deploy-as=root:${mysql_password}; /bin/sleep 3 ; /usr/bin/cloud-setup-management ; /bin/sleep 3 ;
/usr/bin/cloud-setup-databases cloud:cloud@localhost --deploy-as=root:${mysql_password} ; /bin/sleep 3 ; /usr/bin/cloud-setup-management",

creates  => '/var/lib/mysql/cloud',
require  => Package["cloud-client"],
}

service { "cloud-management":
ensure   => "running",
enable   => "true",
require  => Exec["cloud-setup"],
}
}
}

MySQL server manifest


# Class myslqserver
#
# This class installs cloudstack
#
# Author: Oliver Leach
# Date : 20th March 2013
#
# Actions:
# Installs the mysql server package
#
# This includes, the mysql-server package
#
# Requires:
# Centos - other operating systems are not supported by this manifest
#
# Sample Usage:
# include mysql::server
#

class mysql::server ($mysql_password) {

case $::operatingsystem {
"centos": {
$supported = true
}
}

if ($supported == true) {

file { "flag-files" :

ensure     => directory,
path       => "/var/lib/puppet/flags/",
owner      => root,
group      => root,
mode       => "0644",
}

package { "mysql-server":
ensure     => installed,
require    => File["flag-files"],
}

file { "/etc/my.cnf" :
owner      => "root",
group      => "root",
source     => "puppet:///modules/mysql/my.cnf",
require    => Package["mysql-server"],
}

service { "mysqld" :
enable     => true,
ensure     => running,
require    => File["/etc/my.cnf"],
}

$mysql_server_exec_1 => "0000000000001"
$mysql_server_file_1 => "/var/lib/puppet/flags/mysql_server_exec_1"

exec { "restart-mysqld" :
command    => "/sbin/service mysqld restart ; echo \"$mysql_server_exec_1\" ; \"$mysql_server_file_1\"",
unless     => "/usr/bin/test \"`/bin/cat $mysql_server_file_1; /dev/null`\" = \"$mysql_server_exec_1\"",
require    => Service["mysqld"],
}

exec { "set-mysql-password" :
unless     => "/usr/bin/mysqladmin -uroot -p${mysql_password} status",
command    => "/usr/bin/mysqladmin -uroot password ${mysql_password}",
require    => Exec["restart-mysqld"],
}

$mysql_server_exec_2 => "0000000000001"
$mysql_server_file_2 => "/var/lib/puppet/flags/mysql_server_exec_2"

exec { "remove-test-dbs" :
command    => "/usr/bin/mysql -u root -p${mysql_password} -e \"DELETE FROM mysql.db WHERE Db='test' OR Db='test%';\"; echo \"$mysql_server_exec_2\" >> "$mysql_server_file_2\"",

unless     => "/usr/bin/test \"`/bin/cat $mysql_server_file_2 /dev/null`\" = \"$mysql_server_exec_2\"",
require    => Exec["set-mysql-password"],
}

$mysql_server_exec_3 => "0000000000003"
$mysql_server_file_3 => "/var/lib/puppet/flags/mysql_server_exec_3"

exec { "remove-remote-root-access" :
command     => "/usr/bin/mysql -u root -p${mysql_password} -e \"DELETE FROM mysql.user WHERE User='root' AND Host NOT IN ('localhost', '127.0.0.1', '::1'); \" echo \"$mysql_server_exec_3\"; \ "$mysql_server_file_3\"",
unless      => "/usr/bin/test \"`/bin/cat $mysql_server_file_3 /dev/null`\" = \"$mysql_server_exec_3\"",
require     => Exec["set-mysql-password"],
}
}
}

Apache manifest


class apache {

$snat_ip   = $snat_ipaddress

case $operatingsystem {
'CentOS', 'RedHat' : {
$supported = true
}
}

if ($supported == true) {

file { 'httpd_conf_d' :
ensure     => directory,
path       => '/etc/httpd/conf.d/',
recurse    => true,
purge      => true,
force      => true,
owner      => "root",
group      => "root",
mode       => 0644,
require    => Package['httpd'],
}

file { 'cloudstack_conf' :

path       => '/etc/httpd/conf.d/cloudstack.conf',
owner      => root,
group      => root,
mode       => '0644',
content    => template('apache/cloudstack.conf.erb'),
require    => File['httpd_conf_d'],
}

package { 'httpd' :
ensure     => present,
}

package { 'mod_ssl' :
ensure     => present,
}

service { 'httpd' :
ensure     => 'running',
enable     => true,
require    => File['cloudstack_conf'],
}
}
}

and the ERB template:


LoadModule ssl_module modules/mod_ssl.so

NameVirtualHost *:80

ServerName
DocumentRoot /var/www/html/
Redirect permanent / https:///client

Listen 443
NameVirtualHost *:443

SSLEngine On
#SSLCertificateFile /var/lib/puppet/ssl/certs/ca.pem
#SSLCertificateKeyFile /var/lib/puppet/ssl/certs/cert001.pem

SSLCertificateFile /etc/pki/tls/certs/localhost.crt
SSLCertificateKeyFile /etc/pki/tls/private/localhost.key

RewriteEngine On
ProxyRequests On
ProxyVia On

Order deny,allow
Deny from all
Allow from all

Redirect permanent / /client

ProxyPass /client http://localhost:8080/client
ProxyPassReverse /client http://localhost:8080/client

And the iptables manifest


class iptables {

$ipv4_file        => $operatingsystem ? {
"debian"          => '/etc/iptables/rules.v4',
/(RedHat|CentOS)/ => '/etc/sysconfig/iptables',
}

exec { "purge default firewall":
command     => "/sbin/iptables -F && /sbin/iptables-save >

$ipv4_file /sbin/service iptables restart",
onlyif      => "/usr/bin/test `/bin/grep \"Firewall configuration

written by\" $ipv4_file | /usr/bin/wc -l` -gt 0",
user        => 'root',
}

/* Make the firewall persistent */
exec { "persist-firewall":
command     => "/bin/echo \"# This file is managed by puppet. Do not modify manually.\" $ipv4_file /sbin/iptables-save $ipv4_file",
refreshonly => true,
user        => 'root',
}

/* purge anything not managed by puppet */
resources { 'firewall':
purge      => true,
}

firewall { '000 INPUT allow related and established':
state      => ['RELATED', 'ESTABLISHED'],
action     => 'accept',
proto      => 'all',
}

firewall { "001 accept all icmp requests":
proto      => 'icmp',
action     => 'accept',
}

firewall { '002 INPUT allow loopback':
iniface    => 'lo',
chain      => 'INPUT',
action     => 'accept',
}

firewall { '100 allow ssh':
state      => ['NEW'],
dport      => '22',
proto      => 'tcp',
action     => 'accept',
}
firewall { '100 allow port 80':
state      => ['NEW'],
dport      => '80',
proto      => 'tcp',
action     => 'accept',
}
firewall { '100 allow port 443':
state      => ['NEW'],
dport      => '443',
proto      => 'tcp',
action     => 'accept',
}
firewall { '100 allow 9090':
state      => ['NEW'],
dport      => '9090',
proto      => 'tcp',
action     => 'accept',
}

firewall { '100 allow 8250':
state      => ['NEW'],
dport      => '8250',
proto      => 'tcp',
action     => 'accept',
}

firewall { '100 allow 7080':
state      => ['NEW'],
dport      => '7080',
proto      => 'tcp',
action     => 'accept',
}

firewall { '100 allow 8080':
state      => ['NEW'],
dport      => '8080',
proto      => 'tcp',
action     => 'accept',
}

firewall { "998 deny all other requests":
action     => 'reject',
proto      => 'all',
reject     => 'icmp-host-prohibited',
}

firewall { "999 deny all other requests":
chain     => 'FORWARD',
action    => 'reject',
proto     => 'all',
reject    => 'icmp-host-prohibited',
}
}