I have setup NSX and trying to deploy VIO2.0 and integrate with NSX.
Deployment wizard was run without issue but error shown in the middle of the deployment.
Checking the log files on VIO management node, I found the below errors in ansible.log
2016-03-22 18:29:37,689 p=366 u=jarvis | TASK: [config-controller | update neutron lbaas configuration] ****************
2016-03-22 18:29:38,024 p=366 u=jarvis | changed: [172.31.17.157]
2016-03-22 18:29:38,039 p=366 u=jarvis | changed: [172.31.17.158]
2016-03-22 18:29:38,054 p=366 u=jarvis | TASK: [config-controller | initialize neutron database] ***********************
2016-03-22 18:31:20,877 p=366 u=jarvis | failed: [172.31.17.157] => {"changed": true, "cmd": ["neutron-db-manage", "--config-file", "/etc/neutron/neutron.conf", "--config-file", "/etc/neutron/plugins/vmware/nsxv.ini", "upgrade", "head"], "delta": "0:01:42.597801", "end": "2016-03-22 10:31:20.839305", "rc": 1, "start": "2016-03-22 10:29:38.241504", "warnings": []}
2016-03-22 18:31:20,877 p=366 u=jarvis | stderr: Traceback (most recent call last):
File "/usr/bin/neutron-db-manage", line 10, in <module>
sys.exit(main())
File "/usr/lib/python2.7/dist-packages/neutron/db/migration/cli.py", line 238, in main
CONF.command.func(config, CONF.command.name)
File "/usr/lib/python2.7/dist-packages/neutron/db/migration/cli.py", line 105, in do_upgrade
run_sanity_checks(config, revision)
File "/usr/lib/python2.7/dist-packages/neutron/db/migration/cli.py", line 229, in run_sanity_checks
script_dir.run_env()
File "/usr/lib/python2.7/dist-packages/alembic/script.py", line 382, in run_env
util.load_python_file(self.dir, 'env.py')
File "/usr/lib/python2.7/dist-packages/alembic/util.py", line 241, in load_python_file
module = load_module_py(module_id, path)
File "/usr/lib/python2.7/dist-packages/alembic/compat.py", line 79, in load_module_py
mod = imp.load_source(module_id, path, fp)
File "/usr/lib/python2.7/dist-packages/neutron/db/migration/alembic_migrations/env.py", line 109, in <module>
run_migrations_online()
File "/usr/lib/python2.7/dist-packages/neutron/db/migration/alembic_migrations/env.py", line 90, in run_migrations_online
engine = session.create_engine(neutron_config.database.connection)
File "/usr/lib/python2.7/dist-packages/oslo_db/sqlalchemy/session.py", line 413, in create_engine
_test_connection(engine, max_retries, retry_interval)
File "/usr/lib/python2.7/dist-packages/oslo_db/sqlalchemy/session.py", line 591, in _test_connection
six.reraise(type(de_ref), de_ref)
File "<string>", line 2, in reraise
oslo_db.exception.DBConnectionError: (OperationalError) (2003, 'Can\'t connect to MySQL server on \'172.31.17.151\' (111 "Connection refused")') None None
2016-03-22 18:31:20,878 p=366 u=jarvis | FATAL: all hosts have already failed -- aborting
I have checked the IP showed in the log "172.31.17.151" is one of the load balancer and I can connect to the IP through SSH without issue from the management node.
Would anyone could help on this issue? Thanks a lot.
Lewis
what are .157 and .158 VMs?
make sure those ip are not already under use.
in the same log file do you see any other errors related to those IPs?
Please try to command 'telnet 172.31.17.151 3306' to check if the mysql port is monitoring and IP address is not dup.
.157 and .158 VM are the VIO Controller VM
Checked the two IPs are not in use by others.
And seems there is no other error related to these IPs
I have uploaded the full log files to below link
http://gofile.me/1UvHi/fCPog82O
Thanks
I found 17.151 is quite strange that I have defined 17.151-17.168 as management Network and 18.151-18.152 as API Network and 18.153 as Public Virtual IP.
But 17.151 is assigned to VIO-LoadBalancer-0 (from VM Summary) and can be PING but not SSH or telnet.
On VIO-LoadBalancer-0 NIC adapter, I can only see 2 interface which carry 17.163 and 18.151 but not 17.151.
Did you deployed on this environment before, or have previous unsuccessful provision.
I think the /etc/hosts file get corrupted, it has invalid records.
You can delete the failed deployment and clean the /etc/hosts on management server, try to deploy again.
We have fixed this by force refresh /etc/hosts right after the VM is created in VIO 2.5 release.
Yes, I tried to deploy before and was unsuccessful.
I tried to clean up the /etc/hosts and deploy again though same error found ..