I need some help figuring out what is wrong with this service. It is set to start automatically, but never has after a reboot. If I start it manually, it will stay started for 9 minutes, and then enter into a stopped state. After it enters into a stopped state the following error is shown in the event viewer on the vCenter server:
The VMware vSphere Profile-Driven Storage Service service terminated with service-specific error Incorrect function..
Any ideas on where to start to troubleshoot this issue? A google search doesn't return anything that is of much use.
Did you ever resolve this issue? I have just upgraded to vCenter 5.0 Update 3 and have the same issue.]
If I roll back my snapshots to update 2 the issue is not there.
jrmunday... is the web client installed on the VC server too? I've just seen this on a VC5.0U3 install where the Profile-Driven Storage Service would start fine but shut down moments after..
I stopped the web client service and started the Profile-Driven Storage Service again..... now it kept running. Configured the web client service with the delayed start option so that the Profile-Driven Storage Service would have a chance to come up first...
Rebooted a few times to make sure that it worked...all good.
/Rubeck
Hi Rubeck,
Thanks very much, this solves the problem. I was pretty close to finding this as I had traced this java process (using port 31000) to the Web Client;
INFO | jvm 1 | 2013/10/31 12:47:45 | 12:47:45 INFO - Starting SPS HTTP Server
INFO | jvm 1 | 2013/10/31 12:47:45 | 2013-10-31 12:47:45.675:INFO::Logging to STDERR via org.mortbay.log.StdErrLog
INFO | jvm 1 | 2013/10/31 12:47:45 | 12:47:45 INFO - Starting server on [HTTP:0.0.0.0:31000, HTTPS:0.0.0.0:31100]
INFO | jvm 1 | 2013/10/31 12:47:45 | 2013-10-31 12:47:45.706:INFO::jetty-6.1.26.1
INFO | jvm 1 | 2013/10/31 12:47:45 | 2013-10-31 12:47:45.753:WARN::failed SocketConnector@0.0.0.0:31000: java.net.BindException: Address already in use: JVM_Bind
INFO | jvm 1 | 2013/10/31 12:47:45 | 2013-10-31 12:47:45.784:INFO::Started VlsiSslSocketConnector@0.0.0.0:31100
INFO | jvm 1 | 2013/10/31 12:47:45 | 2013-10-31 12:47:45.784:WARN::failed Server@25deb1ad: java.net.BindException: Address already in use: JVM_Bind
INFO | jvm 1 | 2013/10/31 12:47:45 | 12:47:45 ERROR - Storage Policy Service could not be initialized: java.net.BindException: Address already in use: JVM_Bind
INFO | jvm 1 | 2013/10/31 12:47:45 | WrapperSimpleApp:
INFO | jvm 1 | 2013/10/31 12:47:45 | WrapperSimpleApp: Encountered an error running main:
INFO | jvm 1 | 2013/10/31 12:47:45 | WrapperSimpleApp: com.vmware.sps.fault.SpsInitializedException: Storage Policy Service could not be initialized.
INFO | jvm 1 | 2013/10/31 12:47:45 | WrapperSimpleApp: Caused by: java.net.BindException: Address already in use: JVM_Bind
Annoyingly, the only error logged in the system event log is the following;
The VMware vSphere Profile-Driven Storage Service service terminated with service-specific error Incorrect function..
Thanks again for your help.
Cheers,
Jon
Thanks again for your help.
Actually I'm the one thanking you as your trace output shows why this is happening... I needed to get on with my life quickly so I simply started to disable add-ons which isn't a part of the default vCenter install.. Once it worked, I didn't really have the time to figure out what the reason was.
/Rubeck
just faced the same issue. Thanks Rubeck
solution as described.
set the web client on delayed start (or disable if you don't use it)
and start the profile driven storage....
thank you vmware for making me thing again additionally :smileygrin:
I also ran into the same problem and an alternate resolution I got from support was to change the config file for the profile service to change the ports it uses.
Open the wrapper.conf file located at C:\Program Files\VMware\Infrastructure\vSphere Web Client\DMServer\bin\service\conf\
Add these entries to the file then restart the vCenter, profile-driven storage, and web client services.
#restrict port range
wrapper.jvm.port=31550
wrapper.jvm.port.min=31550
wrapper.jvm.port.max=31999
wrapper.port=32550
wrapper.port.min=32550
wrapper.port.max=32999
Hi, After upgrading to vSphere 5.0 U3, we now have the same port grabbing issue across many vCS systems. Any restart, such as simple Windows update, is a time consuming re-check and restart of these services as others have reported.
After testing per the instructions above, changing the WEB CLIENT's wrapper.conf control file solves our restart issue. We did not change the Profile Driven Storage wrapper.conf file.
The PDS wrapper.conf is file's ports are shown below for reference only. Do not change this file. -> C:\Program Files\VMware\Infrastructure\Profile-Driven Storage\conf
# wrapper JVM port
wrapper.jvm.port=31300
wrapper.jvm.port.min=31300
wrapper.jvm.port.max=31999
# wrapper port
wrapper.port=32300
wrapper.port.min=32300
wrapper.port.max=32999
The WEB CLIENT's wrapper.conf needs to have its ports restricted so change this file -> C:\Program Files\VMware\Infrastructure\vSphere Web Client\DMServer\bin\service\conf\
# restrict JVM port range for ESXi 5.0 U3 port race condition
wrapper.jvm.port=31550
wrapper.jvm.port.min=31550
wrapper.jvm.port.max=31999
wrapper.port=32550
wrapper.port.min=32550
wrapper.port.max=32999
# <EOF>
Thanks SO much for the tip. This really needs to make it into the associated KB article which says to just manually restart the services which is not practical. G. Mobley
Can anyone provide the KB for this issue? Seeing the same thing in the lab with VC 5.0 after updating to U3.
There was no single KB when I posted the above in Oct 2013. There were several KB dancing around issues but no solution - maybe there is now...I have not searched lately. G. Mobley
In the vCenter 5.0 U3 release notes (VMware vCenter Server 5.0 Update 3 Release Notes) there is the following paragraph:
I have been trying to sort Profile-Driven Storage service starting and then stopping after updating to 5.0 U3 and found this discussion. Unfortunately we don't have web client, so that wasn't our issue. I eventually tracked it down to our NetApp VSC using the port 31000 (Used netstat -ao, found 31000, made note of PID then found it in Task Manager) and thus not allowing Profile-Driven Storage service to start. I stopped this service, restart the Profile service and then restarted the VSC service. Both services are now running fine. I have subsequently selected a delayed start to the VSC service but not had a chance to reboot to see if it works.
paulnmck - Yep, that did turn out to be the cause of my issue as well (did not realize it was happening even when at 5.0 U2). The delayed start workaround for the VSC service appeared to work for us.
I experienced this same issue recently and it drove me bonkers! I was able to resolve my issue by identifying which service on my vCenter server was causing a port conflict; ultimately identify which service was using port 31000. The first thing I did was run netstat -o to identify where 31000 was being used. On the far right you have a PID column. Use the PID number to identify which process (i.e. Task Manager) is using that port. In my case, I have vSphere Update Manager installed on my vCenter server in my lab environment. The Update Manager service was ultimately the culprit using port 31000.
To resolve the issue I ensured the Profile-Drive Storage Service was set to Automatic startup. I reset the vSphere Update Manager service and vSphere Web Client service to Automatic (Delayed Startup). I rebooted the system to test my settings and the vSphere Profile-Driven Storage Service started successfully.
I pushed the issue with VMware. Please check KB2074505
VMware KB: VMware vSphere Profile-Driven Storage service fails to start after rebooting vCenter S...
I am also trying to get them to edit the command to use netstat -aon | find "31000" instead. that provides a cleaner output.
I had the same issue. NetApp VSC was the culprit in my situation.
I tried paulnmck's solution from this thread which worked until a reboot. To get mine working after reboot I had to adjust the sps-config file (KB2001804)
for the profile-driven storage service.
I increased the # attempts from 10 to 20 and decreased the interval 60 to 10 (seconds) between attempts
attemptNumber = 20
sleepInterval = 100000(milliseconds)
So far so good, all services start successfully after a reboot.
In our situation, it kept rolling around. First VSC, then vCO, then vCO config... whichever one of 5 services that use Java and who's port range is set to start with 31000. Every reboot would bring a new service starting before PDS and stealing 31000.
My advice; fix PDS... not the other 5 services. it is the one forced to use 31000 instead of a dynamic range.