Hi all
I tried to upgrade vRA from 7.3.0 to 7.4 but got the following error:
Last Install: | Failed to install updates(Error while running post-install scripts) on Tuesday, 2018 April 17 23:09:58 UTC+2 VA-check: finished Pre-install: finished Post-install: failed Update failed (code 0-2). Check logs in /opt/vmware/var/log/vami or retry update later. |
In the /opt/vmware/var/log/vami/updatecli.log I found this:
NOTICE: Found extension citext and it exists in pg_catalog
Can not read the Orchestrator pid from /var/lib/vco/app-server/logs/tomcat.pid
+ res=2
+ echo 'Script /etc/bootstrap/postupdate.d/20-vcac failed, error status 2'
+ exit 2
+ rm -f /tmp/postupdate-err-log
+ exit 2
+ trapfunc
+ excode=2
+ test 2 -gt 0
+ vami_update_msg set post-install 'Post-install: failed'
+ test -x /usr/sbin/vami-update-msg
+ /usr/sbin/vami-update-msg set post-install 'Post-install: failed'
+ sleep 1
+ test 2 -gt 0 -o 0 -gt 0
+ vami_update_msg set update-status 'Update failed (code 0-2). Check logs in /opt/vmware/var/log/vami or retry update later.'
+ test -x /usr/sbin/vami-update-msg
+ /usr/sbin/vami-update-msg set update-status 'Update failed (code 0-2). Check logs in /opt/vmware/var/log/vami or retry update later.'
+ exit
17/04/2018 19:09:55 [ERROR] Failed with exit code 512
17/04/2018 19:09:55 [INFO] Update status: Running VMware tools reconfiguration
17/04/2018 19:09:55 [INFO] Running /opt/vmware/share/vami/vami_reconfigure_tools
vmware-toolbox-cmd is /usr/bin/vmware-toolbox-cmd
Configuring VAMI VMware tools service wrapper.
17/04/2018 19:09:55 [INFO] Update status: Done VMware tools reconfiguration
17/04/2018 19:09:55 [INFO] Update status: Error while running post-install scripts
17/04/2018 19:09:55 [ERROR] Failure: updatecli exiting abnormally
17/04/2018 19:09:55 [INFO] Install Finished
Any idea what's wrong?
Thanks,
Andi
I was able to successfully upgrade today from 7.3.0 to 7.4.0 with VMware support.
We have done the following steps:
Soon there should be public KB for this issue.
Hope this helps.
Andi
Select Statement:
select * from (
select categoryid, name, count(*)-1, 'vmo_configelement' as type from vmo_configelement group by categoryid, name having count(*) > 1
union
select parentcategoryid, name, count(*)-1, 'vmo_configelementcategory' as type from vmo_configelementcategory group by parentcategoryid, name having count(*) > 1
union
select elementid, '' as name, count(*)-1, 'vmo_configelementcontent' as type from vmo_configelementcontent group by elementid having count(*) > 1
union
select '' as id, name, count(*)-1, 'vmo_package' as type from vmo_package group by name having count(*) > 1
union
select packageid, '' as name, count(*)-1, 'vmo_packagecontent' as type from vmo_packagecontent group by packageid having count(*) > 1
union
select policyid, '' as name, count(*)-1, 'vmo_policycontent' as type from vmo_policycontent group by policyid having count(*) > 1
union
select categoryid, name, count(*)-1, 'vmo_resourceelement' as type from vmo_resourceelement group by categoryid, name having count(*) > 1
union
select elementid, '' as name, count(*)-1, 'vmo_resourceelementcontent' as type from vmo_resourceelementcontent group by elementid having count(*) > 1
union
select scriptmodulecategoryid, name, count(*)-1, 'vmo_scriptmodule' as type from vmo_scriptmodule group by scriptmodulecategoryid, name having count(*) > 1
union
select '' as id, name, count(*)-1, 'vmo_scriptmodulecategory' as type from vmo_scriptmodulecategory group by name having count(*) > 1
union
select scriptmoduleid, '' as name, count(*)-1, 'vmo_scriptmodulecontent' as type from vmo_scriptmodulecontent group by scriptmoduleid having count(*) > 1
union
select dunesobjectid, version, count(*)-1, 'vmo_versionhistorycontent' as type from vmo_versionhistorycontent group by dunesobjectid, version having count(*) > 1
union
select workflowid, '' as name, count(*)-1, 'vmo_workflowcontent' as type from vmo_workflowcontent group by workflowid having count(*) > 1
) as u;
browse logs under /var/log/bootstrap (esp. "postupdate.log") for more details ... post here.
The postupdate logs:
/var/log/bootstrap # tail -n 100 postupdate.log
SSLOPTS: ?sslmode=verify-ca&sslrootcert=/var/vmware/vpostgres/current/.postgresql/root.crt
Orchestrator's root folder: /var/lib/vco
Orchestrator Configuration Tool. Version: 7.4.0.8071781 Build: 8071781
Can not read the Orchestrator pid from /var/lib/vco/app-server/logs/tomcat.pid
Start configuring database settings.
Configuring database settings finished successfully.
Start updating database scheme.
ch.dunes.vso.db.check.CheckDuplicatesException:
Duplicate entries found in the Orchestrator database:
Resource element duplicates:
1 item with ID '8a8115675c67e982015c67eb4a4403e1' and name 'zh_MO.properties'
1 item with ID '8a8115675c67e982015c67eb4a4403e1' and name 'ja_JP.properties'
1 item with ID '8a8115675c67e982015c67eb4a4403e1' and name 'en.properties'
1 item with ID '8a8115675c67e982015c67eb4a4403e1' and name 'ja.properties'
1 item with ID '8a8115675c67e982015c67eb4a4403e1' and name 'es_ES.properties'
1 item with ID '8a8115675c67e982015c67eb4a4403e1' and name 'zh_CN.properties'
1 item with ID '8a8115675c67e982015c67eb4a4403e1' and name 'fr_FR.properties'
1 item with ID '8a8115675c67e982015c67eb4a4403e1' and name 'es.properties'
1 item with ID '8a8115675c67e982015c67eb4a4403e1' and name 'zh_Hant.properties'
1 item with ID '8a8115675c67e982015c67eb4a4403e1' and name 'de_DE.properties'
1 item with ID '8a8115675c67e982015c67eb4a4403e1' and name 'zh_HK.properties'
1 item with ID '8a8115675c67e982015c67eb4a4403e1' and name 'zh.properties'
1 item with ID '8a8115675c67e982015c67eb4a4403e1' and name 'zh_TW.properties'
1 item with ID '8a8115675c67e982015c67eb4a4403e1' and name 'zh_Hans.properties'
1 item with ID '8a8115675c67e982015c67eb4a4403e1' and name 'en_US.properties'
1 item with ID '8a8115675c67e982015c67eb4a4403e1' and name 'ko_KR.properties'
1 item with ID '8a8115675c67e982015c67eb4a4403e1' and name 'ko.properties'
1 item with ID '8a8115675c67e982015c67eb4a4403e1' and name 'fr.properties'
1 item with ID '8a8115675c67e982015c67eb4a4403e1' and name 'de.properties'
Resolve duplicates by deleting unnecessary items.
java.lang.RuntimeException: ch.dunes.vso.db.check.CheckDuplicatesException:
Duplicate entries found in the Orchestrator database:
Resource element duplicates:
1 item with ID '8a8115675c67e982015c67eb4a4403e1' and name 'zh_MO.properties'
1 item with ID '8a8115675c67e982015c67eb4a4403e1' and name 'ja_JP.properties'
1 item with ID '8a8115675c67e982015c67eb4a4403e1' and name 'en.properties'
1 item with ID '8a8115675c67e982015c67eb4a4403e1' and name 'ja.properties'
1 item with ID '8a8115675c67e982015c67eb4a4403e1' and name 'es_ES.properties'
1 item with ID '8a8115675c67e982015c67eb4a4403e1' and name 'zh_CN.properties'
1 item with ID '8a8115675c67e982015c67eb4a4403e1' and name 'fr_FR.properties'
1 item with ID '8a8115675c67e982015c67eb4a4403e1' and name 'es.properties'
1 item with ID '8a8115675c67e982015c67eb4a4403e1' and name 'zh_Hant.properties'
1 item with ID '8a8115675c67e982015c67eb4a4403e1' and name 'de_DE.properties'
1 item with ID '8a8115675c67e982015c67eb4a4403e1' and name 'zh_HK.properties'
1 item with ID '8a8115675c67e982015c67eb4a4403e1' and name 'zh.properties'
1 item with ID '8a8115675c67e982015c67eb4a4403e1' and name 'zh_TW.properties'
1 item with ID '8a8115675c67e982015c67eb4a4403e1' and name 'zh_Hans.properties'
1 item with ID '8a8115675c67e982015c67eb4a4403e1' and name 'en_US.properties'
1 item with ID '8a8115675c67e982015c67eb4a4403e1' and name 'ko_KR.properties'
1 item with ID '8a8115675c67e982015c67eb4a4403e1' and name 'ko.properties'
1 item with ID '8a8115675c67e982015c67eb4a4403e1' and name 'fr.properties'
1 item with ID '8a8115675c67e982015c67eb4a4403e1' and name 'de.properties'
Resolve duplicates by deleting unnecessary items.
at com.vmware.o11n.integration.initialization.DatabaseInitializer.initializeSchema(DatabaseInitializer.java:77)
at com.vmware.o11n.configuration.database.install.DatabaseInstallerImpl.installDatabase(DatabaseInstallerImpl.java:30)
at com.vmware.o11n.cli.configuration.commands.db.DbCommand.execute(DbCommand.java:100)
at com.vmware.o11n.cli.configuration.commands.ConfigurationCommand.executeCmd(ConfigurationCommand.java:103)
at com.vmware.o11n.cli.configuration.ConfigurationCli.executeCommand(ConfigurationCli.java:143)
at com.vmware.o11n.cli.configuration.ConfigurationCli.main(ConfigurationCli.java:121)
Caused by: ch.dunes.vso.db.check.CheckDuplicatesException:
Duplicate entries found in the Orchestrator database:
Resource element duplicates:
1 item with ID '8a8115675c67e982015c67eb4a4403e1' and name 'zh_MO.properties'
1 item with ID '8a8115675c67e982015c67eb4a4403e1' and name 'ja_JP.properties'
1 item with ID '8a8115675c67e982015c67eb4a4403e1' and name 'en.properties'
1 item with ID '8a8115675c67e982015c67eb4a4403e1' and name 'ja.properties'
1 item with ID '8a8115675c67e982015c67eb4a4403e1' and name 'es_ES.properties'
1 item with ID '8a8115675c67e982015c67eb4a4403e1' and name 'zh_CN.properties'
1 item with ID '8a8115675c67e982015c67eb4a4403e1' and name 'fr_FR.properties'
1 item with ID '8a8115675c67e982015c67eb4a4403e1' and name 'es.properties'
1 item with ID '8a8115675c67e982015c67eb4a4403e1' and name 'zh_Hant.properties'
1 item with ID '8a8115675c67e982015c67eb4a4403e1' and name 'de_DE.properties'
1 item with ID '8a8115675c67e982015c67eb4a4403e1' and name 'zh_HK.properties'
1 item with ID '8a8115675c67e982015c67eb4a4403e1' and name 'zh.properties'
1 item with ID '8a8115675c67e982015c67eb4a4403e1' and name 'zh_TW.properties'
1 item with ID '8a8115675c67e982015c67eb4a4403e1' and name 'zh_Hans.properties'
1 item with ID '8a8115675c67e982015c67eb4a4403e1' and name 'en_US.properties'
1 item with ID '8a8115675c67e982015c67eb4a4403e1' and name 'ko_KR.properties'
1 item with ID '8a8115675c67e982015c67eb4a4403e1' and name 'ko.properties'
1 item with ID '8a8115675c67e982015c67eb4a4403e1' and name 'fr.properties'
1 item with ID '8a8115675c67e982015c67eb4a4403e1' and name 'de.properties'
Resolve duplicates by deleting unnecessary items.
at ch.dunes.vso.db.check.ValidateDb.checkForDuplicates(ValidateDb.java:126)
at com.vmware.o11n.integration.initialization.DatabaseInitializer.initializeSchema(DatabaseInitializer.java:74)
... 5 more
Error updating the DB.
Script /etc/bootstrap/postupdate.d/20-vcac failed, error status 2
Seems like some duplicate entries at the Orchestrator database 😕
I'll circle back with the team and report. Please open an SR with vmware support for tracking!
Thanks, I will open a SR with VMware support.
I´m facing the same issue. Please share the solution if there is one.
You can open Java client and remove duplicates before upgrade of vRA.
You need to navigate to resource elements view in designer mode and examine the structure. You should check what of the duplicates is used and delete the other one.
Hope this helps
According to VMware support engineering is still trying to find the root cause of this issue.
Hopefully they will have some news by Monday/Tuesday.
what ivand recommends should help;
rootcause is really duplicate elements in vRO / vRO DB. Those should be cleared manually before the upgrade.
question: have you used the "Code Stream Management Pack for IT DevOps (Houdini)" at some point in the past?
No, we haven´t used Houdini.
This was caused because you have duplicate resource elements and configuration items. Most likely from importing packages (I've seen it happen a few times).
Find each of these items (there will be more than 1 each) and delete the extras. Then re-run the install.
I was able to successfully upgrade today from 7.3.0 to 7.4.0 with VMware support.
We have done the following steps:
Soon there should be public KB for this issue.
Hope this helps.
Andi
Select Statement:
select * from (
select categoryid, name, count(*)-1, 'vmo_configelement' as type from vmo_configelement group by categoryid, name having count(*) > 1
union
select parentcategoryid, name, count(*)-1, 'vmo_configelementcategory' as type from vmo_configelementcategory group by parentcategoryid, name having count(*) > 1
union
select elementid, '' as name, count(*)-1, 'vmo_configelementcontent' as type from vmo_configelementcontent group by elementid having count(*) > 1
union
select '' as id, name, count(*)-1, 'vmo_package' as type from vmo_package group by name having count(*) > 1
union
select packageid, '' as name, count(*)-1, 'vmo_packagecontent' as type from vmo_packagecontent group by packageid having count(*) > 1
union
select policyid, '' as name, count(*)-1, 'vmo_policycontent' as type from vmo_policycontent group by policyid having count(*) > 1
union
select categoryid, name, count(*)-1, 'vmo_resourceelement' as type from vmo_resourceelement group by categoryid, name having count(*) > 1
union
select elementid, '' as name, count(*)-1, 'vmo_resourceelementcontent' as type from vmo_resourceelementcontent group by elementid having count(*) > 1
union
select scriptmodulecategoryid, name, count(*)-1, 'vmo_scriptmodule' as type from vmo_scriptmodule group by scriptmodulecategoryid, name having count(*) > 1
union
select '' as id, name, count(*)-1, 'vmo_scriptmodulecategory' as type from vmo_scriptmodulecategory group by name having count(*) > 1
union
select scriptmoduleid, '' as name, count(*)-1, 'vmo_scriptmodulecontent' as type from vmo_scriptmodulecontent group by scriptmoduleid having count(*) > 1
union
select dunesobjectid, version, count(*)-1, 'vmo_versionhistorycontent' as type from vmo_versionhistorycontent group by dunesobjectid, version having count(*) > 1
union
select workflowid, '' as name, count(*)-1, 'vmo_workflowcontent' as type from vmo_workflowcontent group by workflowid having count(*) > 1
) as u;
BRILLIANT WORK anba89
I had the same problem...however it wasn't a plugin in my case but an Config element that had the same name but was in differnt folders. Removing the config elemnet and cleaning up its dependencies (all workflows that used that conifg element) solved the problem.
Ours was similar but not plugin related. We are not using Horizon plugin but are using the embedded vIDM which has some Horizon code so it wasn't applicable. Here is the order of events we did with VMware Technical Support:
KB reference link: Upgrade vRealize Automation 7.3.0 to 7.4 failed
1. We captured the /var/log/bootstrap on primary appliance after failure.
Portion relevent within file:
Configuration element duplicates:
1 item with categoryid '8a7dc2326158633d01617737063140e4' and name 'InstanceDetails'
1 item with categoryid '8a7dc2326158633d01617737063140e4' and name 'Allocation'
Configuration element category duplicates:
1 item with parentcategoryid 'null' and name 'Azure'
2. Reverted snaps
3. Putty into primary vRA VA as root
4. Ran following commands:
su postgres
/opt/vmware/vpostgres/current/bin/psql
\c vcac
\dt
5. In our case, the following tables were affected:
vmo_configelement
vmo_configelementcategory
----Performed following for vmo_configelement table----
a. Ran the following command to capture the appropriate column names:
select * from vmo_configelement;
b. Ran the following command:
select id, categoryid, name from vmo_configelement order by name;
We saw the second duplicate IDs for names InstanceDetails and Allocation:
fae18e53-79b8-4a2e-8da9-c3353d288b98
c8b1645a-e4a1-430b-a6a3-eae4b98109e5
c. Ran the following commands to delete from the vmo_configelement table:
delete from vmo_configelement where id='fae18e53-79b8-4a2e-8da9-c3353d288b98';
delete from vmo_configelement where id='c8b1645a-e4a1-430b-a6a3-eae4b98109e5';
d. Ran the following command to confirm deletion from the vmo_configelement table:
select id, categoryid, name from vmo_configelement order by name;
----Performed following for vmo_configelementcategory table----
aa. Ran the following command for the vmo_configelementcategory table to capture the appropriate column names:
select * from vmo_configelement;
bb. Ran the following command for the vmo_configelementcategory table:
select id, categoryid, name from vmo_configelement order by name;
We saw the second duplicate id for name Azure:
8a7dc2326158633d01617737063140e4
cc. Ran the following commands to delete from the vmo_configelementcategory table:
delete from vmo_configelementcategory where id='8a7dc2326158633d01617737063140e4' and name='Azure';
dd. Ran the following command to confirm deletion from thevmo_configelementcategory table:
select id, categoryid, name from vmo_configelementcategory order by name;
----Table cleanup complete
Note: You can install and execute the RPM (listed in KB) and should receive the following:
The command does not need database configuration update.
Rerun the update