Cloudstack

Cloudstack db.properties password encryption

I was once asked to update the db password in the db.properties files in Cloudstack. I kind of knew the guy who asked me was still in Cloudstack 2.2 land where passwords were in plain old text, but as of Cloudstack 3.x, the passwords in db.properties file and other passwords for that fact are encrypted. Lets have a closer look at the db.properties file.

The following file on each cloudstack and cloudplatform server contains the db connection details

/etc/cloud/management/db.properties

In this file there is information about the db username and password details.

# CloudStack database tuning parameters
db.cloud.password=ENC(57nNfvJaN9X54lbJi0pjugc9YylyRo8c)
db.cloud.username=cloudadmin
db.cloud.host=10.0.26.16
db.cloud.validationQuery=SELECT 1
db.cloud.testOnBorrow=true

In cloudplatform and cloudstack version 3 +, the db.cloud.password is encrypted. To change this, run the following file to work out the encrypted password

java -classpath /usr/share/java/cloud-jasypt-1.8.jar 
org.jasypt.intf.cli.JasyptPBEStringEncryptionCLI 
encrypt.sh input="Password12" 
password="$(cat /etc/cloud/management/key)" verbose=true

Where

input=Password12

is the password you wish to encrypt.

The output should look something like this.

$ java -classpath /usr/share/java/cloud-jasypt-1.8.jar org.jasypt.intf.cli.JasyptPBEStringEncryptionCLI encrypt.sh input="Password12" password="$(cat /etc/cloud/management/key)" verbose=true

----ENVIRONMENT-----------------

Runtime: Sun Microsystems Inc. OpenJDK 64-Bit Server VM 20.0-b12

----ARGUMENTS-------------------

verbose: true
input: Password12
password: password

----OUTPUT----------------------

57nNfvJaN9X54lbJi0pjugc9YylyRo8c

The 57nNfvJaN9X54lbJi0pjugc9YylyRo8c is the encrypted password for Password12. Use this as the new password in the db.properties file

If you wish to decrypt the password, run the following command:

java-classpath/usr/share/java/cloud-jasypt-1.8.jar org.jasypt.intf.cli.JasyptPBEStringDecryptionCLI decrypt.shinput="57nNfvJaN9X54lbJi0pjugc9YylyRo8c"  password="$(cat /etc/cloud/management/key)"verbose=false

The password is then displayed, in this case Password12

Cloudstack deploying cloudstack

Image

I was talking at the Cloudstack European Meet up the other day about getting Cloudstack to deploy Cloudstack.

Well its more than that – I was talking about the concepts rather than what I was doing, as really, Cloudstack deploying Cloudstack is a bit of a gimick.

Here is my github link to the project
https://github.com/oliverleach/cloudstack-puppet

However, I thought I would include some scribble on how the project works in case anyone is interested:

Cloudstack 4.0.1 deploy

This project enables a running Cloudstack instance 3.x and above to deploy Cloudstack 4.0.1 instances. This could be to demo Cloudstack 4.0.1 or used for testing purposes, by easily spinning up Cloudstack 4.0.1 instances. It could also be used to provision private clouds using Cloudstack.

How to use this project

The python script and puppet manifests in this repo deploys cloudstack 4.0.1 using Puppet and the Cloudstack API. For this to work, you need to use the python scripts and 2 puppet templates. One template is the puppet-master, and one template is the puppet-agent.

You need to configure the python parameters in the puppet-deploy.py script as per your environment. This includes:

1 – your API key
2 – your secret key
3 – your service offering
4 – your Cloudstack zone ID
5 – your network ID of your user account
6 – The Cloudstack provisioning host IP and port
7 – The Cloudstack protocol used (http or https)

You will also need your 2 template IDs for the puppet master and puppet agent

These templates can be downloaded using the following link:

Puppet Agent

https://dl.dropbox.com/u/79905244/puppet-agent.7z

Puppet Master

https://dl.dropbox.com/u/79905244/puppet-master.7z

Both templates are running CentOS 6.4 64-bit. The puppet master is configured as a puppet-ca, a puppet server and a puppet agent. The puppet agent has the puppet agent installed and runs a userdata.rb script on startup. See the rc.local file for further information.

Once the templates are unzipped and uploaded in to your provisioning Cloudstack instance, update the puppet-deploy.py script with the template IDs.

To run the python scripts, use an interpreter like idle, open and run the puppet-deploy.py file, or simply run from the command line where an instance of python is installed; e.g. python puppet-deploy.py. This has been tested against python 2.7.3.

The git repo holds the puppet manifests which are included in the puppet-master template.
They are just for reference.

This project is just for fun, could be better written and I hold no responsibility if you use the project. You are allowed to do so.

Puppet manifests for cloudstack and co.

Here are some of my puppet manifests for Cloudstack and co. You can also find more information here:

https://github.com/oliverleach/cloudstack-puppet

I also recently did a presentation at the Cloudstack European meetup in London on the 11th April about deploying applications using puppet and Cloudstack. The talk was well received so I thought I would jot down the manifests that I wrote as part of the talk.

Here is the Cloudstack manifest:


# Class cloudstack
#
# This class installs cloudstack
#
# Author: Oliver Leach
# Date : 18th March 2013
#
# Actions:
# Installs the cloudstakc package
#
# This includes, the cloudstack-client, mysql-server and any install commands
#
# Currently the clodustack class is monlithic for demo purposes.
#
# Requires:
# Centos - other operating systems are not supported by this manifest
#
# Sample Usage:
# include cloudstack
#

class cloudstack ($mysql_password) {

case $::operatingsystem {

'centos': {
$supported = true
}
}

if ($supported == true) {

yumrepo { 'cloudstack':
baseurl  => "http://cloudstack.apt-get.eu/rhel/4.0/",
descr    => "Cloudstack 4.0.1 yum repo",
enabled  => 1,
gpgcheck => 0,
}

package { "cloud-client":
ensure   => "installed",
require  => Yumrepo["cloudstack"],
}

exec { "cloud-setup" :
command  => "/usr/bin/cloud-setup-databases cloud:cloud@localhost --deploy-as=root:${mysql_password}; /bin/sleep 3 ; /usr/bin/cloud-setup-management ; /bin/sleep 3 ;
/usr/bin/cloud-setup-databases cloud:cloud@localhost --deploy-as=root:${mysql_password} ; /bin/sleep 3 ; /usr/bin/cloud-setup-management",

creates  => '/var/lib/mysql/cloud',
require  => Package["cloud-client"],
}

service { "cloud-management":
ensure   => "running",
enable   => "true",
require  => Exec["cloud-setup"],
}
}
}

MySQL server manifest


# Class myslqserver
#
# This class installs cloudstack
#
# Author: Oliver Leach
# Date : 20th March 2013
#
# Actions:
# Installs the mysql server package
#
# This includes, the mysql-server package
#
# Requires:
# Centos - other operating systems are not supported by this manifest
#
# Sample Usage:
# include mysql::server
#

class mysql::server ($mysql_password) {

case $::operatingsystem {
"centos": {
$supported = true
}
}

if ($supported == true) {

file { "flag-files" :

ensure     => directory,
path       => "/var/lib/puppet/flags/",
owner      => root,
group      => root,
mode       => "0644",
}

package { "mysql-server":
ensure     => installed,
require    => File["flag-files"],
}

file { "/etc/my.cnf" :
owner      => "root",
group      => "root",
source     => "puppet:///modules/mysql/my.cnf",
require    => Package["mysql-server"],
}

service { "mysqld" :
enable     => true,
ensure     => running,
require    => File["/etc/my.cnf"],
}

$mysql_server_exec_1 => "0000000000001"
$mysql_server_file_1 => "/var/lib/puppet/flags/mysql_server_exec_1"

exec { "restart-mysqld" :
command    => "/sbin/service mysqld restart ; echo \"$mysql_server_exec_1\" ; \"$mysql_server_file_1\"",
unless     => "/usr/bin/test \"`/bin/cat $mysql_server_file_1; /dev/null`\" = \"$mysql_server_exec_1\"",
require    => Service["mysqld"],
}

exec { "set-mysql-password" :
unless     => "/usr/bin/mysqladmin -uroot -p${mysql_password} status",
command    => "/usr/bin/mysqladmin -uroot password ${mysql_password}",
require    => Exec["restart-mysqld"],
}

$mysql_server_exec_2 => "0000000000001"
$mysql_server_file_2 => "/var/lib/puppet/flags/mysql_server_exec_2"

exec { "remove-test-dbs" :
command    => "/usr/bin/mysql -u root -p${mysql_password} -e \"DELETE FROM mysql.db WHERE Db='test' OR Db='test%';\"; echo \"$mysql_server_exec_2\" >> "$mysql_server_file_2\"",

unless     => "/usr/bin/test \"`/bin/cat $mysql_server_file_2 /dev/null`\" = \"$mysql_server_exec_2\"",
require    => Exec["set-mysql-password"],
}

$mysql_server_exec_3 => "0000000000003"
$mysql_server_file_3 => "/var/lib/puppet/flags/mysql_server_exec_3"

exec { "remove-remote-root-access" :
command     => "/usr/bin/mysql -u root -p${mysql_password} -e \"DELETE FROM mysql.user WHERE User='root' AND Host NOT IN ('localhost', '127.0.0.1', '::1'); \" echo \"$mysql_server_exec_3\"; \ "$mysql_server_file_3\"",
unless      => "/usr/bin/test \"`/bin/cat $mysql_server_file_3 /dev/null`\" = \"$mysql_server_exec_3\"",
require     => Exec["set-mysql-password"],
}
}
}

Apache manifest


class apache {

$snat_ip   = $snat_ipaddress

case $operatingsystem {
'CentOS', 'RedHat' : {
$supported = true
}
}

if ($supported == true) {

file { 'httpd_conf_d' :
ensure     => directory,
path       => '/etc/httpd/conf.d/',
recurse    => true,
purge      => true,
force      => true,
owner      => "root",
group      => "root",
mode       => 0644,
require    => Package['httpd'],
}

file { 'cloudstack_conf' :

path       => '/etc/httpd/conf.d/cloudstack.conf',
owner      => root,
group      => root,
mode       => '0644',
content    => template('apache/cloudstack.conf.erb'),
require    => File['httpd_conf_d'],
}

package { 'httpd' :
ensure     => present,
}

package { 'mod_ssl' :
ensure     => present,
}

service { 'httpd' :
ensure     => 'running',
enable     => true,
require    => File['cloudstack_conf'],
}
}
}

and the ERB template:


LoadModule ssl_module modules/mod_ssl.so

NameVirtualHost *:80

ServerName
DocumentRoot /var/www/html/
Redirect permanent / https:///client

Listen 443
NameVirtualHost *:443

SSLEngine On
#SSLCertificateFile /var/lib/puppet/ssl/certs/ca.pem
#SSLCertificateKeyFile /var/lib/puppet/ssl/certs/cert001.pem

SSLCertificateFile /etc/pki/tls/certs/localhost.crt
SSLCertificateKeyFile /etc/pki/tls/private/localhost.key

RewriteEngine On
ProxyRequests On
ProxyVia On

Order deny,allow
Deny from all
Allow from all

Redirect permanent / /client

ProxyPass /client http://localhost:8080/client
ProxyPassReverse /client http://localhost:8080/client

And the iptables manifest


class iptables {

$ipv4_file        => $operatingsystem ? {
"debian"          => '/etc/iptables/rules.v4',
/(RedHat|CentOS)/ => '/etc/sysconfig/iptables',
}

exec { "purge default firewall":
command     => "/sbin/iptables -F && /sbin/iptables-save >

$ipv4_file /sbin/service iptables restart",
onlyif      => "/usr/bin/test `/bin/grep \"Firewall configuration

written by\" $ipv4_file | /usr/bin/wc -l` -gt 0",
user        => 'root',
}

/* Make the firewall persistent */
exec { "persist-firewall":
command     => "/bin/echo \"# This file is managed by puppet. Do not modify manually.\" $ipv4_file /sbin/iptables-save $ipv4_file",
refreshonly => true,
user        => 'root',
}

/* purge anything not managed by puppet */
resources { 'firewall':
purge      => true,
}

firewall { '000 INPUT allow related and established':
state      => ['RELATED', 'ESTABLISHED'],
action     => 'accept',
proto      => 'all',
}

firewall { "001 accept all icmp requests":
proto      => 'icmp',
action     => 'accept',
}

firewall { '002 INPUT allow loopback':
iniface    => 'lo',
chain      => 'INPUT',
action     => 'accept',
}

firewall { '100 allow ssh':
state      => ['NEW'],
dport      => '22',
proto      => 'tcp',
action     => 'accept',
}
firewall { '100 allow port 80':
state      => ['NEW'],
dport      => '80',
proto      => 'tcp',
action     => 'accept',
}
firewall { '100 allow port 443':
state      => ['NEW'],
dport      => '443',
proto      => 'tcp',
action     => 'accept',
}
firewall { '100 allow 9090':
state      => ['NEW'],
dport      => '9090',
proto      => 'tcp',
action     => 'accept',
}

firewall { '100 allow 8250':
state      => ['NEW'],
dport      => '8250',
proto      => 'tcp',
action     => 'accept',
}

firewall { '100 allow 7080':
state      => ['NEW'],
dport      => '7080',
proto      => 'tcp',
action     => 'accept',
}

firewall { '100 allow 8080':
state      => ['NEW'],
dport      => '8080',
proto      => 'tcp',
action     => 'accept',
}

firewall { "998 deny all other requests":
action     => 'reject',
proto      => 'all',
reject     => 'icmp-host-prohibited',
}

firewall { "999 deny all other requests":
chain     => 'FORWARD',
action    => 'reject',
proto     => 'all',
reject    => 'icmp-host-prohibited',
}
}

Cloudstack 4.1 new features

new fatures

Here is a list I have put together which discusses the Cloudstack 4.1. new features. I have put some comments down against some of the features. Some comments are missing, simply because I do not know much about them!

  • AWS-Style Regions

This is the biggest change which involves placing a new layer above Zones. This is so we can scale cloudstack better with the ability of having a cloudstack management server per availablity zone. This is great if you customers are geographically apart across continents. There are some big challenges around this change I suspect.

  • F5 & SRX inline mode

This is an improvement. Basically having a firewall in front of the F5 load balancer so your customer can have the advantage of IDP and IDS filtering.

  • AutoScale in the Netscalers

This has been in CloudPlatform and requires NS firmware 10+, but its cool and it works. You do need a netscaler, and the cheapest option is a VPX device.

  • Advanced Search UI

Required as the product is becoming so more powerful, with the ability to handle so many devices now. Tick!

  • S3-backed Secondary Storage

Well S3 has about 15% of the worlds entire storage, so why not 🙂

  • EC2 Query API support

Well EC2 has about 15% of the worlds entire compute force, so why not 🙂

  • API Request Throttling

This has been outstanding for a while. You can actually DDOS a CSM server if you wanted to. This will stop the potential.

  • Enhanced baremetal servers support on Cisco UCS

This is a good. Automating physical hosts with Cisco UCS platform, allowing Cloudstack to manage the lifecycle of the physical host.

  • Events framework to publish/subscribe to CloudStack events

Good for developers who want to hook in to events. Can be used for emailing when something happens which isn’t really possible in CS 4

  • OVS support in KVM

I would say this is a pit of plumbing for some future networking functionality coming our way with KVM. A step in the right direction.

  • Reset SSH Key to access VM (similar to reset password)

Great stuff. Easy and now usable.

  • Security Groups Isolation in Advanced Zone

I am sure this is only KVM using etables but I have also heard this is coming in to XenServer at some point when using Advanced Zones

  • Site-to-Site VPN: Monitoring of VPN Tunnels 
  • Persistent Networks without running a VM

This is actually fairly important if using cloudstack to burst or scale into. You have vlan dedicated to you so persistent config can now be applied to your external devices connecting in to cloudstack.

  • Egress firewall rules for guest network

I need to see this. This is in CloudPlatform but we need to have more control over outbound traffic, i.e., what we can allow through a back door. For example, if I wanted to connect to a kms server for licensing or a monitoring server for my customers. However, it is worth noting that if the KMS port is blocked by default, it will still retry and hopefully someone opens up the KMS port. It is a problem though in my opinion until we can a back door setting that controls what ports are allows outbound by default. If in doubt, you can script this – get in touch if you want more information

  • Additional VMX Settings

Kelvin Yang, the Citrix Developer for VMware integration in Cloudstack, often finds a few gems to put in the vmx file. This could be passthrough or vmxnet3 drivers or something to do with link clones and snaps.

  • Resize volumes feature

This is root and data volumes. Great if you have deployed a VM and want to resize it.

  • Add/remove network on VM

Yes – and finally. So you need a NIC? You can now add on.

  • Limit API Queries

1,2,3,4,5 that’s all you can have Mr.

  • Improve API Performance / Add Search Capabilities

With so many APIs now in Cloudstack, we can get a bit lost with it all. This will help.

  • Allow for same vlan on different physical nics

Helpful if you want to define your networks a bit more over physical devices. For example, coming inbound via an MPLS link.

  • BigSwitch network plugin

I get lost with all of the SDNs coming our way. This one I don’t know too much about, but someone else does http://blogs.citrix.com/2012/11/13/the-big-switch-to-sdn-and-cloudstack-guest-post-from-bigswitch/

  • ApiDiscoverService: Implement a plugin mechanism that exposes the list of APIs through a discovery service on the management server

Oh, this one again. So what was the other one then?

  •  Implement L3 Router functionality in Nicira Nvp Plugin

No more vlans.. no more vlans.. yeah.

  • Mash up marvin into an interactive auto-completing API shell for CloudStack (aka Cloudmonkey CLI)

  • Baremetal kickstart

Good for physical host deployments of say Windows 🙂

  • Nicira NVP/KVM

SDN not my game.. yet – I need it to settle down a bit more before I get a grip 🙂

  • Netscaler plugin

More netscaler integration. I mist admit, having implemented the SDX devices, there was a lot of work that these guys needed to do to get this better. There was no option for uploading SSL certs or SSL offloading, or creating traffic policies – all which are important features I would like to see in Cloudstack and CloudPlatform. They will come based on demand.

  • API changes for IPv6 Support IP6 stuff coming in to play more.

Lots of new features. Can’t wait to have a play and will report back when I do. Leave a comment if you get the chance to muck around – be good to know feedback!

Using the cloudstack API

I’ve recently been working on a project to enhance a testing strategy that involved revisiting cloudstack API. I dove headfirst into the deep end, which made me realise just how much you can do with it. I’d like to share some of my experiences, in the hope of getting some feedback from people who have used CloudStack or CloudPlatform or are thinking about it and want to find out more. After plunging into the depths of the API, this is what I discovered.

First, it’s good! Everything you can do in our compute platform can be done via the API and it’s actually quite simple once you get the hang of things. I wrote my test scripts in Python, but there are a few different languages you can use and I know some guys have dived in using PHP and .NET. There is also a Java library available from Jclouds so you can make your choice depending on your comfort zone.

Second, most of the API commands are asynchronous, depending on what you’re trying to do. This means you can call an API command and move on to the next one or, like me, hold out for the async command response. This way you can do one task, wait for the response and grab certain criteria for the next command. You can then build up a fairly comprehensive set of commands. For example, you can deploy a virtual machine (VM) using the deployVirtualMachine together with the following:

  • Serviceofferingid: The serviceofferingid relates to what instance size you wish to use. You can get a list of available service offerings by using the listServiceOfferingscommand.
  • Templateid: The templateid refers to the ID of either the templates we offer to customers, or one you have preconfigured. These values can be obtained by using the listTemplates command.
  • Zoneid: The zoneid signifies the zone in which you deploy the VM. This is achieved by running the listZones command.

The good news is that once you have these values, you can use them over and over again. Or you can list all the template IDs and service offerings and pass a value from a collection if you want different size instances or want to use different templates.

Once you have built up your deployVirtualMachine API command string, you’re ready to move on to the next command – and this is just the beginning. As the deployVirtualMachine is an async command, what you really need before you can move on is the ID of the VM you have deployed.

Then you can do other things like attach disks if required, assign an IP or enable a static NAT, as these commands need a VM ID to work. This is where the queryAsyncJobResult command comes into play. Once you run your command to deploy a VM, it responds with a jobid and jobstatus. The jobid is the asyncjobid number, which you can query using the queryAsyncJobResult command. Once the deployVirtualMachine job finishes, the jobstatus changes to 1, which means you can find out all sorts of information about the VM that has been deployed. One key piece of information is the VM ID, which is required for most commands to work. Once you have this, the world is your oyster.

What I did was create a class in Python, which I was able to reuse to find out the status of the async job. Once I had this working, I was able to call the method in the class every time I wanted to find out the status of the async job.

So, let’s quickly look at attaching a disk to a virtual machine. First, you need to create a disk using createVolume and, again, apply the queryAsyncJobResult to find out the ID of the volume you have created. You will also use the diskofferingid to create a disk of a certain size (I created a 50GB disk). Once you have this information and the queryAsyncJobResult comes back, you are ready to attach the disk to the VM. Here are the required API commands to do so:

The virtualmachineid is the ID of the VM, which you can get when using the queryAsyncJobResult of the deployVirtualMachine job.

ID refers to the ID of the disk volume you can obtain when using the queryAsyncJobResult of the createVolume.

Once you have built up your createVolume and have the corresponding async job results, you’re ready to use the attachVolume commands. Hey presto! You’ve now deployed a VM and attached a disk. And so it continues…

This isn’t a tutorial or documentation explaining how to use our API but merely a blog on how I did it. I’ve just finished testing all our API commands and yes, there are many ways to skin a cat, but the point is that it can be done and you can achieve a lot with just the click of a button. It’s not all plain sailing – it never is – but once you get involved, you can easily work these things out and it does become quite simple.

I’m keen to hear from people who have used our API and what programming language they used, or from people who are thinking about using the API its capabilities. I’m happy to share some of the framework of the Python scripts I’ve put together, so get in touch if you’d like a hand getting started with our compute API.