VM Deployment fails

[ashish_S]

recently I have updated the certificates of VRA. soon after the certificate update I have been getting error below.

The following component requests failed: RHEL7.x. Request failed: a: sendEBSMessage15(workflow=d103aa6f-a1c2-4b4b-8c13-50656f6dae54) Error in state VMPSMasterWorkflow32.BuildingMachine phase PRE event (queue = ac02ebef-090c-4f66-b7ff-3e81b2664408): Extensibility consumer error(20999) - TypeError: Cannot read property "keys" from undefined (Dynamic Script Module name : propertiesToJson#4), ght6484: sendEBSMessage4(workflow=d103aa6f-a1c2-4b4b-8c13-50656f6dae54) Error in state VMPSMasterWorkflow32.Requested phase PRE event (queue = 9c4ebe2f-825b-4450-901f-cdad877459b5): Extensibility consumer error(20999) - TypeError: Cannot read property "keys" from undefined (Dynamic Script Module name : propertiesToJson#4), and alight6484: sendEBSMessage9(workflow=d103aa6f-a1c2-4b4b-8c13-50656f6dae54) Error in state VMPSMasterWorkflow32.WaitingToBuild phase PRE event (queue = 6444a757-fbab-420a-a599-32eb0a2c05dc): Extensibility consumer error(20999) - TypeError: Cannot read property "keys" from undefined (Dynamic Script Module name : propertiesToJson#4).

VRA Version is 7.6

[ronaldod]

As it says : "keys" from undefined 

The variable keys is not defined.

The thing is what event is giving the problem. And is this only with one type of deployment ?

[CallistoJag]

Have you renewed the connection from vRO to vRA after updating the certs?

[CallistoJag]

and to any other resources in the environment, as most connections are linked to certs 🙂

[ashish_S]

Yes I did, I followed the article https://kb.vmware.com/s/article/74573. Its embedded installation

[ashish_S]

the issue is will all the blueprints and all of them were working before the upgrage

[emacintosh]

It's certainly possible that something changed with the certificate update, but maybe that's also just a coincidence.  I'd treat this like a normal troubleshooting issue.

It's telling you where the problem is: the VMPSMasterWorkflow32.BuildingMachine workflow.  And i'm not sure if it's a scriptable task in that workflow or an action being called, but somewhere you have an element called propertiesToJson.  In there, on line 4, is where your error is occurring. 

I would start there.  What does that line of code do?  Assuming somwhere in there is an someObject.keys type reference, and you'd want to ensure that the someObject is actually a thing, defined, etc.  Does that script make possibly incorrect assumptions.

If you can't figure it out at a glance, I'd start testing.  For us, that would mean making sure the workflow ends before trying to do anything impactful and then rerunning failed workflow runs.  With those, ust normal troubleshooting:  confirm it fails, see what someObject actually is/contains, etc.  

If you get stuck, maybe you can share some more details with us so we can help with the troubleshooting effort.  But for now, just go figure out what it's complaining about.  Maybe it's something obvious like some inventory in vRO needs updated to reflect to the new certs somehow, and so the object you're getting in some propertiesToJson is undefined instead of containing an actual object.

[ashish_S]

Thanks for replying. I noticed one thing, when I opened the VRO there is a workflow (workflow Dispatcher) which is getting failed. I am adding the screenshots let me know if you have any clue about it. 

[emacintosh]

Ok, it's been a while since we were on 7.x, and I didn't use it for that long myself.  But I'm guessing something in that scriptable task from the first screenshot is not doing what is expected.  Is that a homegrown workflow?  Or is that something that just gets called by the event broker in 7.x automatically?

In any case, something in there is ultimately relying on propertiesToJson.  And whatever is getting passed to it is not valid. 

I would personally start adding some logging in there if possible.  For example, maybe log something in the for loop to see if that is working as expected.  And also log the __asd_xxx vars being created, since they're immediately used.  You're going to need better eyes into what is actually happening before you can figure out how to fix it, and logging is a simple way to get started.

And if you ever find the propertiesToJson action/script/whatever, it could be helpful to know if it's homegrown.  Because maybe it just needs some additional error checking to handle null or undefined objects better?

[ashish_S]

I found propertiestoJson. I'm attaching the screenshot of it.

Below is the complete script 

I found propertiestoJson. I'm attaching the screenshot of it.

var ObjectWrapper = System.getModule("com.vmware.pscoe.library.util").ObjectWrapper();

var variables = new Properties();

function addProperties(source, target) {

for (var index in target.keys) {

source.put(target.keys[index], target.get(target.keys[index]));

}

}

if (vCACVm && vCACVm.getEntity()) {

addProperties(variables, vCACVm.getEntity().getProperties());

}

addProperties(variables, vCACVmProperties);

var propertiesWrapper = new ObjectWrapper({});

for (var index in vCACVmProperties.keys) {

var propVal = null;

if (vCACVmProperties.get(vCACVmProperties.keys[index]) != null) {

var propValStr = System.getModule("com.vmware.pscoe.library.util")

.processTemplate(vCACVmProperties.get(vCACVmProperties.keys[index]), variables);

try {

var parsedValue = JSON.parse(propValStr);

//stringified json should not be parsed back to objects 

if(typeof parsedValue !== 'object') {

propVal = parsedValue;

} else {

propVal = propValStr;

}

} catch(e) {

//Failed to parse it might contain special chars so treat as a string

propVal = propValStr;

}

var objNames = vCACVmProperties.keys[index].split(".");

propertiesWrapper.addValueWithPath(objNames, propVal);

}

}

return propertiesWrapper.obj;