vRealize Orchestrator 7 VAPI vSphere tags workflow

I created a workflow using the VAPI plugin to assign vSphere tags to a VC:VirtualMachine object.

You will need to import the package, then configure VAPI with your vCenter endpoints by following these instructions:

1 – Run the Import VAPI metamodel workflow

2 – Run the Add VAPI endpoint workflow

Both workflows take the same inputs, but you must run the workflows in that order. Screen shot of inputs as follows:

Screen Shot 2016-07-19 at 22.56.04

 

Once you have configured your VAPI endpoints, download the following package from github

com.virtualdevops.tags.package

Import the package and then the following workflows and actions.

Screen Shot 2016-07-19 at 23.02.25

Screen Shot 2016-07-19 at 23.03.11

Run the Add tag to VC vm workflow (highlighted in the package screen shot above), and as long as you have configured VAPI endpoints, you can add vSphere Tags and vSphere Categories to a VC:VirtualMachine object.

10_26_38_40

  • The vCenter Virtual Machine is the VC:VirtualMachine object
  • The VAPI Endpoint drop down list should contain all your VAPI endpoints
  • The Create new Tag Category allows you to create a new Category
  • The Tag Category displays existing tag categories configured on your VAPI Endpoint
  • The Tag Name is the name of the tag you wish to create
  • The Tag description is the description and is optional

Code is more of an example, as I found the OOTB VAPI plugins for tags pretty limited. You can move the input parameters to attributes as desired and pass in values for the tag name using maybe an API call to a SNOW or CMDB to get values for your tag name.

Any issues, send in a comment.

 

vRealize Cloud Client pagesize configuration

When running the vRealize Cloud Client CLI tool, the default page size is set to 25. This means that any command you run will return the 25 items on a page. Sometime you may have more than 25 items, for example for xaas-blueprints

Screen Shot 2016-06-27 at 09.25.14

You can include this parameter to set the pagesize

vra catalog list --pageSize 50

This will return 50 items per page.

However, you can set the default page size in your cloudclient configuration by configuring the cloudclient.config file on the virtual machine you are running cloud client on. For example:

vi ~/.cloudclient/cloudclient.config

And here is an example configuration

Screen Shot 2016-06-27 at 09.29.31

default.page.size can be set to any number you want.

 

 

 

Creating multiple log files using Python logging library

I was looking at the Python logging library as I needed to create 2 log files for different logging purposes. I ended up creating the following Python log functions.

#!/usr/bin/env python
import logging

LOG_FILE_ONE = "/var/log/one.log"
LOG_FILE_TWO = "/var/log/two.log"

def main():

    setup_logger('log_one', LOG_FILE_ONE)
    setup_logger('log_two', LOG_FILE_TWO)

    logger('Logging out to log one...', 'info', 'one')
    logger('Logging out to log two...', 'warning', 'two')

def setup_logger(logger_name, log_file, level=logging.INFO):

    log_setup = logging.getLogger(logger_name)
    formatter = logging.Formatter('%(levelname)s: %(asctime)s %(message)s', datefmt='%m/%d/%Y %I:%M:%S %p')
    fileHandler = logging.FileHandler(log_file, mode='a')
    fileHandler.setFormatter(formatter)
    streamHandler = logging.StreamHandler()
    streamHandler.setFormatter(formatter)
    log_setup.setLevel(level)
    log_setup.addHandler(fileHandler)
    log_setup.addHandler(streamHandler)

def logger(msg, level, logfile):
 
    if logfile == 'one'   : log = logging.getLogger('log_one')
    if logfile == 'two'   : log = logging.getLogger('log_two') 
    if level == 'info'    : log.info(msg) 
    if level == 'warning' : log.warning(msg)
    if level == 'error'   : log.error(msg)

if __name__ == "__main__":

    main()

In the main() function, we set up a logging instance. We do this for both LOG_FILE_ONE and LOG_FILE_TWO.

    setup_logger('log_one', LOG_FILE_ONE)
    setup_logger('log_two', LOG_FILE_TWO)

This uses standard logging python library. The formatter is important as that is how the log entries look in the log file (below). Here is mine:

formatter = logging.Formatter('%(levelname)s: %(asctime)s %(message)s', 
            datefmt='%m/%d/%Y %I:%M:%S %p')

Have a look at this online doc from Python.org for more info on the formatter method.

https://docs.python.org/2/library/logging.html#formatter-objects

Once we set up the logging instance, including the file streamer and handler, we can then use those instances,  in this case log_one or log_two.

Both have different log file paths /var/log/{one.log,two.log}.

We then call the logging function, telling it what logging instance to use (log_one or log_two) and we also pass in what log level we want (info, warn, debug).

    logger('Logging out to log one...', 'info', 'one')
    logger('Logging out to log two...', 'warn', 'two')

Interesting, we get this output when we run this script.

INFO: 01/01/2016 01:01:01 AM Logging out to log one…

Notice we do not get WARNING being logged.

That is because of this:

def setup_logger(logger_name, log_file, level=logging.INFO):

Change level=logging.WARN, and this happens:

INFO: 01/01/2016 01:01:01 AM Logging out to log one...
WARNING: 01/01/2016 01:01:01 PM Logging out to log two...

So, pretty handy and you can create as many log files as you need…

Updating vRA ASD Service Blueprint workflows

There isn’t an easy way to find out what vRealize Orchestrator workflow is associated to a vRealize Automation Service Blueprint. You can query the vPostgres database to find out the workflow ID as follows:

1 – Log in to vPostgres and connect to the vcac database

2 – Run the following SQL command

vcac=# \x on;
Expanded display is on.
vcac=# select * from asd_serviceblueprint;

This should output something similar to this:

-[ RECORD 1 ]------------------------------------------------
id | 850b9e17-024e-44f7-9831-a5651a5d6e0f
description | This is a workflow description.
name | My XAAS workflow
status | 1
tenant | vsphere.local
workflowid | 1eea8b02-15e8-41b8-992e-3df1cd8e4b99
baseform |
outputparameter | ea001bf4-d43f-4c74-a010-b6027c7ecbdf
catalogrequestinfohidden | t
version | 1.0.0
-----------------------------------------------------------

You can change the vRealize Orchestrator workflow that gets called by a vRealize Automation service blueprint pretty easily, by changing the workflow ID value in the vPostgres database

vcac=# update asd_serviceblueprint 
set workflowid='ad9e2cc2-2efa-44c1-a574-6a02bec2f998'
where id = '9d6510ad-24a6-4f0c-adc8-736cc3e99d77';

This will change what workflow gets executed without redeploying the ASD form. It’s handy when you want to clone the ASD service blueprint and actually change the underlying workflow to test something new, without having to rebuild the ASD form.

You can also run a vRealize Orchestrator workflow using the below javascript to find out what the name is using the workflow ID

wfId = '9d6510ad-24a6-4f0c-adc8-736cc3e99d77'
System.log("Workflow name: " + Server.getWorkflowWithId(wfId).name);

 

All of this can be managed by creating your own service blueprint and publishing as a catalog item..

vRealize Automation vPostgres password

The password for vPostgres in vRealize Automation 7 is now encrpyted. This means you will not be able to log in to vPostgres without decrypting the password. To do this, follow these commands:

Check the password name value pair in file : /etc/vcac/server.xml

Eg:
password=”s2enc~Bp/gQ0sSz5ejlemiTxsflCjMNp0GsnHD6tvahuh5Fpw=”

Run the below command on VA to get encrypted password:

vcac-config prop-util -d --p "s2enc~Bp/gQ0sSz5ejlemiTxsflCjMNp0GsnHD6tvahuh5Fpw="

The above command give the password which you can connect to vPostgres.

Then run the following commands:

su postgres
cd /opt/vmware/vpostgres/current/bin
./psql vcac -W

Enter the password and it should let you in to the postgres console, connected to the vcac database.

The other way, is to log in as postgres user and change to the vcac database, but that really is too easy.

./psql postgres
psql.bin (9.4.5 (VMware Postgres 9.4.5.0-3183988 release))
Type "help" for help.
postgres=# \c vcac;
You are now connected to database "vcac" as user "postgres".
vcac=#

 

vRealize 7 Orchestrator endpoint

I’ve just hit a config issue when configuring the embedded vRealize 7 Orchestrator Endpoint in vRealize Automation 7, aka vRA 7. You may hit an error when running a vRealize Orchestrator data collection or hit this error when running a NSX data collection:

Workflow ‘vSphereVCNSInventory’ failed with the following exception:
Endpoint not found. There is no vRealize Orchestrator endpoint configured with property __VMware.VCenterOrchestrator.Plugin.NSX.Build.

In vRealize 6, we used to have to add the endpoint for orchestrator as:

https://vroserver:8281/vco

However vRA now has an embedded LB HAproxy running and you don’t need add port 8281. Therefore the endpoint config just looks like this:

https://vroserver/vco

Screen Shot 2016-01-08 at 10.44.50

One other thing. If you try to enter that endpoint in a browser, it won’t work. You need to append a forward slash, so it should look like this:

https://vroserver/vco/

You will need to add an orchestrator endpoint before being able to run a NSX data collection. You don’t need to configure the NSX plugin and that will get configured as long as your embeeded vRrealize orchestrator endpoint is correct. Additionally, use the load balancer address as required.

 

 

 

 

How to get vCAC:VirtualMachine object and entity from VC:VirtualMachine

A quick post to show you a couple of ways to get the vCAC:VirtualMachine object and the entity from a VC:VirtualMachine object. Note, below I have got the VC:VirtualMachine as an input parameter called vm.

There are a few ways we can achieve this and one way would be to use a filter on Server.findAllForType. Here’s a snippet of code:

// filter on vm.id

var vraVm = Server.findAllForType("vCAC:VirtualMachine", "ExternalReferenceId eq '" + vm.id + "'");

// end of code

This returns an array. I found it is best to use LinqPad to work out what you can filter on. For example, if you filter on the vCAC:VirtualMachine properties, some are different cases and won’t return a result.

You can also filter using an ‘and’. For example:

// Filter using an 'and'

var vraVm = Server.findAllForType("vCAC:VirtualMachine", "IsDeleted eq false and ExternalReferenceId eq '" + vm.id + "'");

//end of code

Notice that I am using IsDeleted, which is what LinqPad defines, rather than isDeleted, which is what the property in the vCAC:VirtualMachine scripting object has defined.

However, the ExternalReferenceId could have multiple values if you have more that one vCenter endpoint configured. Another value you can filter on if you do have multiple VC’s is VMUniqueID.

So, simply as follows:

// filter on vm.config.instanceUuid

var vraVm = Server.findAllForType("vCAC:VirtualMachine", "VMUniqueID eq '" + vm.config.instanceUuid + "'");

// end of code

Notice that we use vm.config.instanceUuid now instead of vm.id. The attribute simply returns a different value.

In this case, vraVm returns an array, as we are doing a findAllForType, which puts the result set into an array. Therefore, we can get the first index to get our vCAC:VirtualMachine object and then use the getEntity() method to get the entity.

// get entity

var vmEntity = vraVm[0].getEntity();

// end of code

Simple stuff. You can use comparison operators, but generally speaking, I find that you are matching of specific values returned from the VC object, vm.id or vm.config.instanceUuid.

You should also check the length of your array as if the array returns more than 1 index, then you know you have multiple matches – can happen when using multiple VC endpoints.

If you return more than one index, then you can do extra checking, like match on the virtual machine name.

However as you may be using this in an action item for a resource action, try to keep down the code evaluation to improve performance.You can use the filter for any other scripting object, for example vCAC:Blueprint and use LinqPad to work out what you can filter on.

Monitoring vRealize Automation services using vRealize Orchestrator

You can list the services and their status on the vRealize Automation appliance by going to the following URL.

https://<vra-fqdn>/component-registry/services/status/current

It returns an XML page with elements containing information about the each service. Here’s a screenshot.

Screen Shot 2015-05-18 at 08.34.16

This gave me an idea as you can poll this URL to check the status of the services using a vRO workflow and email if the service status isn’t returned.

Here’s a basic example of some code to poll the URL in a vRO workflow.


// Create Properties object for reporting any unregistered services
var serviceReport = new Properties();

// Form URL constructor
var vraUrl = new URL("https://<va-fqdn>/component-registry/services/status/current");

// Get content
var content = vraUrl.getContent();

// Parse into JSON object
content = JSON.parse(content);

// Get object values
for ( var v in content.content){

	System.log("Service: " + content.content[v].serviceName + ". Intitialised: " + 
				content.content[v].serviceStatus.initialized + ". Status: " + content.content[v].serviceStatus.serviceInitializationStatus)

	if (content.content[v].serviceStatus.serviceInitializationStatus != "REGISTERED"){
		serviceReport.put(content.content[v].serviceName,content.content[v].serviceStatus.serviceInitializationStatus)
	}	
}

It’s a simple workflow that puts any services into a properties object based on some logic, in this case if not equal to REGISTERED. You could then email a support desk, or could retry and check again, or be really funky and check to see if there is a change control ticket open, and if not, then notify support. The key is once you have your workflow created, then you will need to schedule it to run, say every 5 minutes.

You don’t obviously have to use vRO either and use some enterprise monitoring solution, which is what I would recommend.

vRealize Automation reservation service API

I was recently looking in to the vRealize Automation REST API reservation service. I was using 6.2.1 and I was tying to create a reservation using the REST API. I think it’s actually a good way to do this as, 1 it’s better than using the ODATA API connecting to the ManagementModelEntities.svc, and 2, I think it’ll be the way forward to use the vRA REST API in the future.

The online documentation for doing this is a good place to start but I found there are errors, which are easily overcome. I will come onto this later, but some of the JSON fields are incorrect. Here’s the documentation link I followed: http://pubs.vmware.com/vra-62/index.jsp?topic=%2Fcom.vmware.vra.programming.doc%2FGUID-75940FA3-1C17-451C-86FF-638E02B7E3DD.html

However, before we get to the errors in the documentation, the first thing I had to do it get a token. I did this using the firefox REST API plugin. To get a token, you need to POST a token request as follows, making sure you add the Content-Type : application/JSON & Accept : application/JSON headers.

Screen Shot 2015-04-22 at 20.43.54

This is a typical response – the “id” is the token.

Screen Shot 2015-04-22 at 20.44.16

So, now I have a token, I can put this in an Authroization Header and save to use in my up and coming REST API calls:

Screen Shot 2015-04-22 at 20.49.52

Notice how I have “Bearer MTQy….” at the beginning of the Value section in the header. You must include Bearer + the API auth token value, otherwise your header isn’t recognized.

Now we have a valid token, we can start using the API service. To create a reservation, you have to build up the reservation JSON POST payload. First, you need to find out some constructs that make up the JSON payload to create a reservation, one being finding the reservation types. Here’s a REST API call to do this (notice this is a GET request not a POST request as we are just getting information):

Screen Shot 2015-04-22 at 20.57.57

The GET URL contains reservation-service/api/reservations/types. This is the URI that you need to append to your vRA host to get the reservation types it supports. Also, notice my tokens (Authorization Bearer token header being the most important, but also my Content-Type : application/JSON & Accept : application/JSON headers). Here’s the response:

Screen Shot 2015-04-22 at 20.58.58

Next, I need to get the subtenant ID. This is the business group in my tenant (I am using the default tenant here, vsphere.local). My business group is called cloud. So to get this value, I need to run  the following REST API GET call:

Screen Shot 2015-04-22 at 21.06.14

The GET URL contains identity/api/tenants/vsphere.local/subtenants. This returns the following:

Screen Shot 2015-04-22 at 21.08.12

The ID in the JSON response can be checked using the vRO vCAC plugin. You need this value to put in the POST payload when creating the reservation. Next up, we need to get the compute resource for the reservation. This one is a REST API POST call, as follows:

Screen Shot 2015-04-22 at 21.16.29

Notice the empty payload {}. I just put this in as otherwise my POST REST request didn’t return anything. I got this back:

Screen Shot 2015-04-22 at 21.16.35

The Compute Cluster is my Compute Resource in vRA. So, to recap, I have my bearer token, reservation type, my business group ID and my compute resource ID.

Next up is a bit more involved. I need to get my resource schema for each of the reservation constructs. Looking at the documentation here, I need to get reservationMemory, reservationNetworks,  computeResource etc. This is where I found some documentation errors, which I will show you in a moment. However to get the resourcePool schema, I need to run the following API REST call:

Screen Shot 2015-04-22 at 22.02.01

The URL appended to the vRA host is /reservation-service/api/data-service/schema/Infrastructure.Reservation.Virtual.vSphere/default/resourcePool/values. This gives me the schema payload for the resourcePool. I need to take the underlyingValue.

Screen Shot 2015-04-22 at 22.02.19

I need to do this for all the required constructs, reservationNetworks, reservationMemory, computeResource, machineQuota, reservationStorages, resourcePool.

My FINAL payload for creating the JSON payload to create a reservation looks like this:


{
 "name" : "TestCreateReservation",
 "reservationTypeId" : "Infrastructure.Reservation.Virtual.vSphere",
 "tenantId" : "vsphere.local",
 "subTenantId" : "727a7899-b074-4f0a-8ccf-08962f7945c3",
 "enabled" : true,
 "priority" : 3,
 "alertPolicy" : {
 "enabled" : true,
 "frequencyReminder" : 20,
 "emailBgMgr" : false,
 "recipients" : [],
 "alerts" : [
 {
 "alertPercentLevel" : 10,
 "referenceResourceId" : "storage",
 "id" : "storage"
 },
 {
 "alertPercentLevel" : 20,
 "referenceResourceId" : "memory",
 "id" : "memory"
 },
 {
 "alertPercentLevel" : 30,
 "referenceResourceId" : "cpu",
 "id" : "cpu"
 },
 {
 "alertPercentLevel" : 40,
 "referenceResourceId" : "machine",
 "id" : "machine"
 }
 ]
 },
 "extensionData" : {
 "entries" : [
 {
 "key" : "reservationNetworks",
 "value" : {
 "type" : "multiple",
 "elementTypeId" : "COMPLEX",
 "items" : [
 {
 "type" : "complex",
 "componentTypeId" : "com.mycompany.csp.iaas.blueprint.service",
 "componentId" : null,
 "classId" : "reservationNetwork",
 "typeFilter" : null,
 "values" : {
 "entries" : [
 {
 "key" : "networkPath",
 "value" : {
 "type" : "entityRef",
 "componentId" : null,
 "classId" : "Network",
 "id" : "c086a74b-cda1-44fe-8873-ec647265ed08",
 "label" : "VM Network"
 }
 }
 ]
 }
 }
 ]
 }
 },
 {
 "key" : "reservationMemory",
 "value" : {
 "type" : "complex",
 "componentTypeId" : "com.mycompany.csp.iaas.blueprint.service",
 "componentId" : null,
 "classId" : "reservationMemory",
 "typeFilter" : null,
 "values" : {
 "entries" : [
 {
 "key" : "hostMemoryTotalSizeMB",
 "value" : {
 "type" : "integer",
 "value" : 10240
 }
 },
 {
 "key" : "memoryReservedSizeMb",
 "value" : {
 "type" : "integer",
 "value" : 10240
 }
 }
 ]
 }
 }
 },
 {
 "key" : "computeResource",
 "value" : {
 "type" : "entityRef",
 "componentId" : null,
 "classId" : "ComputeResource",
 "id" : "4ad3b822-1099-4355-91b1-47d4dd8f8a56",
 "label" : "Compute Cluster"
 }
 },
 {
 "key" : "machineQuota",
 "value" : {
 "type" : "integer",
 "value" : 2
 }
 },
 {
 "key" : "reservationStorages",
 "value" : {
 "type" : "multiple",
 "elementTypeId" : "COMPLEX",
 "items" : [
 {
 "type" : "complex",
 "componentTypeId" : "com.mycompany.csp.iaas.blueprint.service",
 "componentId" : null,
 "classId" : "reservationStorage",
 "typeFilter" : null,
 "values" : {
 "entries" : [
 {
 "key" : "storageTotalSizeGB",
 "value" : {
 "type" : "integer",
 "value" : 50
 }
 },
 {
 "key" : "storageReservedSizeGB",
 "value" : {
 "type" : "integer",
 "value" : 50
 }
 },
 {
 "key" : "storageEnabled",
 "value" : {
 "type" : "boolean",
 "value" : true
 }
 },
 {
 "key" : "storagePath",
 "value" : {
 "type" : "entityRef",
 "componentId" : null,
 "classId" : "StoragePath",
 "id" : "48d5b4fc-ab89-4483-ab15-20c0eaab210b",
 "label" : "of-nfs01"
 }
 },
 {
 "key" : "storageFreeSizeGB",
 "value" : {
 "type" : "integer",
 "value" : 50
 }
 },
 {
 "key" : "storageReservationPriority",
 "value" : {
 "type" : "integer",
 "value" : 1
 }
 }
 ]
 }
 }
 ]
 }
 },
 {
 "key" : "resourcePool",
 "value" : {
 "type" : "entityRef",
 "componentId" : null,
 "classId" : "ResourcePools",
 "id" : "79e1f291-1001-4ec3-b0b2-c094f8fc865e",
 "label" : "Resources"
 }
 }
 ]
 }
 }

The errors I got were around the wrong constructs being passed in. I received this message:

{
"errors": [
{
"code": 11001,
"message": "Field with id networkPath is required.",
"systemMessage": "Field with id networkPath is required.",
"moreInfoUrl": null
},
{
"code": 11001,
"message": "Field with id storagePath is required.",
"systemMessage": "Field with id storagePath is required.",
"moreInfoUrl": null
},
{
"code": 11001,
"message": "Field with id storageReservationPriority is required.",
"systemMessage": "Field with id storageReservationPriority is required.",
"moreInfoUrl": null
},
{
"code": 11001,
"message": "Field with id storageEnabled is required.",
"systemMessage": "Field with id storageEnabled is required.",
"moreInfoUrl": null
}
]
}

Once I substituted the fields with the correct ones, my JSON payload was good. For example

storageEnabled = reservationStorageEnabled
storageReservationPriority = reservationStorageReservationPriority

You can get these values from LINQPAD as this is the ODATA entities it is expecting.

Good luck!

vRealize Orchestrator – using Telnet to test port connectvitiy

vRealize Orchestrator has a TelnetClient scripting class that you can use to test port connectivity. This is far easier to use than dropping to a telnet client on a cli and test connectivity that way. Together with using the TelnetClient scripting class and a workflow that would loop and wait for the port to be open, fail gracefully or continue the workflow but notify that connectivity wasn’t available, the options the TelnetClient scripting class gives you are very handy.

Here a screenshot of what the scripting class looks like in the API browser:

Screen Shot 2015-03-08 at 11.19.18

The constructor looks like this:


var telnet = new TelnetClient(string)

The parameter is a string and takes a terminal type, which be default is VT100. Other terminal types would be ANSI, VT52, VT102, VTNT and so on. VVT stands for video terminal so mostly you can use the default value of VT100.

Here is some simple code to test telnet connectivity:


var port = 22; // number
var target = "localhost"; // string

try {

// testing for port connectivity on target host
System.log("Trying to connect to port " + port + " on host  " + target);
var telnet = new TelnetClient("vt100") ;
telnet.connect(target, port);
connectionSuccess = telnet.isConnected();
System.log("Connectivity to target host " + target + " is successful")

} catch (e) {

connectionSuccess = telnet.isConnected();
System.log("Connection failed: " + e)

} finally {

telnet.disconnect()

}

The isConnected() method returns a boolean of true or false, depending on the connection result, so together with this block of code and a loop workflow, you can test connectivity to a telnet port or wait for a socket to be up. Using a try / cath / finally statement, you catch any errors and disconnect appropriately, although strictly speaking disconnec() can go in the try statement as you’ll only ever connect in the try block so you can only ever disconnect there as well. Skin a cat in several ways on that one. Up to you 🙂

Here’s a screenshot of a workflow that puts it all together. The decision just checks for true or false which is set depending on the connection response.

Screen Shot 2015-03-08 at 11.27.05

Obviously in the sample code above, you should configure the inputs appropriately for the target and the port. This is also a good way to check if a service is available, for example check for SMTP or a REST API service by using telnet.

Happy telnetting!

references:

http://goo.gl/f7NqqR
http://goo.gl/v8QZA

workflow:

http://goo.gl/cOLz8d