Vcenter Upgrade issues from 4.1 to 4.1 Update1

[BT08]

Hi All,

Yesterday I have applied the U1 for VCenter. While the upgrade did finish without any error, some of the things in VCenter is now broken:

1) Storage management service (sms) was red and per logs it could not initialize. It was trying to access to http://localhost/sms/smService-web/health.xml. After changing the URL in C:\Program Files\VMware\Infrastructure\VirtualCenter Server\extensions\com.vmware.vim.sms\extension.xml to https://localhos:8443/... , this started working again.

2) 'storage views' tab. It was working before the upgrade. I know this is similar to the issue above but I cannot locate any xml file to change it. Per viclient log,

WebException occured  in Update : System.Net.WebException: The request failed with HTTP status 503: Service Unavailable.

WebException occured in QueryTopology : System.Net.WebException: The request failed with HTTP status 503: Service Unavailable.

WebException occured  in QueryList : System.Net.WebException: The request failed with HTTP status 503: Service Unavailable.

I believe it is probably using port 80. Do you know which file it is trying to access?

3) "licensing reporting manager" is not working either. When clicked, I get Script Error pop up

When I click yes or no:

Then I get

Authentication required error in the licensing manager window.

Is there a way to change this URL too?

4) The last issue I have is that when I go to Vcenter page, http://vcenter, and click on web access I get redirected to http://vcenter/ui with 503 error. Web access is working fine if I use https://localhost:8443/ui. Any ideas what I can do to fix this?

We have used default ports for everything. The server is 2008 R2 with 2008 SQL SE installed on the same server.

I appreaciate any help.

Thank you.

[alefestaedist]

sounds like a webserver error.. did you check if you have any windows log error too? I would check if the webserver is up and running

Alex

[BT08]

Thanks. Web server seems fine. I logged into tomcat manager and all services are up and running. Also other web services must be working as VCenter Service status screen shows all green. :smileyconfused:

[alefestaedist]

Regardin the 503 error you could try to follow this KB http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=100929...

Alex

[BT08]

I followed the article. VPXD is listening over 80, 443, and over several other ports (8085, 8089, 5xxxx etc) . However, I cannot reinstall web access as when I click add/remove, I have two options, 1) configuring linked mode and 2) removing the VCenter.

One more question:

I had taken a ghost image of the server before the upgrade. Can I revert server back to this image? Would this create any issues as I have already pushed new VCenter agents to the ESXi servers to be able to reconnect them?

Thank you.

[alefestaedist]

I would not recomend to revert the image back, but you may eventually reinstall the vcenter as if you were moving it on another machine.

this could help you:http://kb.vmware.com/selfservice/microsites/search.do?cmd=displayKC&docType=kc&externalId=7960893&sl... 0 158654006

but before go through that decision I would try to troubleshoot your problem.

So.. let see if we can fix that.

So can you confirm that you dont'have:

1)firewall running

2) IIS running

3) any other webserver that is running (i.e.:like IBM services do sometimes)?

Alex

[BT08]

Hi Alex,

At this point, I agree, I am not planning to revert the image back instead work through the issues. That would be my last resort. 

1) Firewall is running but I turned it off to test and received the same errors.

2) IIS is not installed.

3) Only tomcat which was installed by VCenter.

The server is only running VCenter and SQL, and management agents, SCCM, SCOM, and backupExec.

Thank you.

[alefestaedist]

humm wonder if could be related to something else.

I would check this:

1) odbc connection

2) sql connection trhough client/odbc

3)restart services and see if throw out some logs

4) vmware logs

Alex

[BT08]

ODBC is fine, I also checked it for possible spaces per another KB and registry keys. I am also able to connect to SQL from remote client. No errors in windows logs.

After restarting web services, I have the following info/warning:

Catalina*.log:

INFO: validateJarFile(C:\Program Files\VMware\Infrastructure\tomcat\webapps\sms\WEB-INF\lib\servlet-api.jar) - jar not loaded. See Servlet Spec 2.3, section 9.7.2. Offending class: javax/servlet/Servlet.class

stats.log:

[2011-03-08 10:19:43,031 Thread-35  WARN  org.apache.commons.httpclient.HttpMethodDirector] Unable to respond to any of these challenges: {cert=CERT realm="SessionManager"}

Not sure if these would help...

[alefestaedist]

humm did you change the jre version lately? maybe an update of java?

Try to remove servlet.jar from your web-inf/lib and you should have it in jakarta-tomcat-xxx\common\lib already

but honestly I am not sure it is related.

hummm did you check that in the tomcat config the java version is correct in case?

Alex

[BT08]

No, I did not change JRE version. When I removed it I got the followings in localhost.log

SEVERE: Malformed moduleBaseURL:
java.net.MalformedURLException: no protocol:
at java.net.URL.<init>(URL.java:567)
at java.net.URL.<init>(URL.java:464)
at java.net.URL.<init>(URL.java:413)
at com.google.gwt.user.server.rpc.RemoteServiceServlet.doGetSerializationPolicy(RemoteServiceServlet.java:196)
at com.google.gwt.user.server.rpc.RemoteServiceServlet.getSerializationPolicy(RemoteServiceServlet.java:116)
at com.google.gwt.user.server.rpc.impl.ServerSerializationStreamReader.prepareToRead(ServerSerializationStreamReader.java:303)
at com.google.gwt.user.server.rpc.RPC.decodeRequest(RPC.java:234)
at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:162)
at com.google.gwt.user.server.rpc.RemoteServiceServlet.doPost(RemoteServiceServlet.java:85)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:857)
at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
at java.lang.Thread.run(Thread.java:619)
Mar 8, 2011 11:01:23 AM org.apache.catalina.core.ApplicationContext log
INFO: ERROR: The module path requested, null, is not in the same web application as this servlet, /vws.  Your module may not be properly configured or your client and server code maybe out of date.
Mar 8, 2011 11:01:23 AM org.apache.catalina.core.ApplicationContext log
INFO: WARNING: Failed to get the SerializationPolicy 'A54E696C43E49725CD8446E4171EA2C4' for module ''; a legacy, 1.3.3 compatible, serialization policy will be used.  You may experience SerializationExceptions as a result.
Mar 8, 2011 11:01:38 AM org.apache.catalina.core.ApplicationContext log
SEVERE: Malformed moduleBaseURL:
java.net.MalformedURLException: no protocol:
at java.net.URL.<init>(URL.java:567)
at java.net.URL.<init>(URL.java:464)
at java.net.URL.<init>(URL.java:413)
at com.google.gwt.user.server.rpc.RemoteServiceServlet.doGetSerializationPolicy(RemoteServiceServlet.java:196)
at com.google.gwt.user.server.rpc.RemoteServiceServlet.getSerializationPolicy(RemoteServiceServlet.java:116)
at com.google.gwt.user.server.rpc.impl.ServerSerializationStreamReader.prepareToRead(ServerSerializationStreamReader.java:303)
at com.google.gwt.user.server.rpc.RPC.decodeRequest(RPC.java:234)
at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:162)
at com.vmware.vim.cimmonitor.gwt.server.GwtCimServiceImplementation.processCall(GwtCimServiceImplementation.java:222)
at com.google.gwt.user.server.rpc.RemoteServiceServlet.doPost(RemoteServiceServlet.java:85)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:857)
at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
at java.lang.Thread.run(Thread.java:619)
Mar 8, 2011 11:01:38 AM org.apache.catalina.core.ApplicationContext log
INFO: ERROR: The module path requested, null, is not in the same web application as this servlet, /vws.  Your module may not be properly configured or your client and server code maybe out of date.
Mar 8, 2011 11:01:38 AM org.apache.catalina.core.ApplicationContext log
INFO: WARNING: Failed to get the SerializationPolicy 'FC7E86DEAEFD2AFEA8F62B491DE5C027' for module ''; a legacy, 1.3.3 compatible, serialization policy will be used.  You may experience SerializationExceptions as a result.

I also have lots of

org.apache.tomcat.util.http.Parameters processParameters
WARNING: Parameters: Invalid chunk '' ignored.

Can you point me to tomcat config file that has java version? Do you know which web service storage view tab is calling? If I can figure it out, I can do some local http calls to see if it is returning anything?

Thank you.

[alefestaedist]

this is mine tomcat config, just consider that is a real standard config without any particular customization.

About the error you got I found this KB that could help:http://kb.vmware.com/selfservice/microsites/search.do?cmd=displayKC&docType=kc&externalId=1030085&sl... 0 158702945

#
# List of comma-separated packages that start with or equal this string
# will cause a security exception to be thrown when
# passed to checkPackageAccess unless the
# corresponding RuntimePermission ("accessClassInPackage."+package) has
# been granted.
package.access=sun.,org.apache.catalina.,org.apache.coyote.,org.apache.tomcat.,org.apache.jasper.,sun.beans.
#
# List of comma-separated packages that start with or equal this string
# will cause a security exception to be thrown when
# passed to checkPackageDefinition unless the
# corresponding RuntimePermission ("defineClassInPackage."+package) has
# been granted.
#
# by default, no packages are restricted for definition, and none of
# the class loaders supplied with the JDK call checkPackageDefinition.
#
package.definition=sun.,java.,org.apache.catalina.,org.apache.coyote.,org.apache.tomcat.,org.apache.jasper.

#
#
# List of comma-separated paths defining the contents of the "common"
# classloader. Prefixes should be used to define what is the repository type.
# Path may be relative to the CATALINA_HOME or CATALINA_BASE path or absolute.
# If left as blank,the JVM system loader will be used as Catalina's "common"
# loader.
# Examples:
#     "foo": Add this folder as a class repository
#     "foo/*.jar": Add all the JARs of the specified folder as class
#                  repositories
#     "foo/bar.jar": Add bar.jar as a class repository
common.loader=${catalina.home}/lib,${catalina.home}/lib/*.jar

#
# List of comma-separated paths defining the contents of the "server"
# classloader. Prefixes should be used to define what is the repository type.
# Path may be relative to the CATALINA_HOME or CATALINA_BASE path or absolute.
# If left as blank, the "common" loader will be used as Catalina's "server"
# loader.
# Examples:
#     "foo": Add this folder as a class repository
#     "foo/*.jar": Add all the JARs of the specified folder as class
#                  repositories
#     "foo/bar.jar": Add bar.jar as a class repository
server.loader=

#
# List of comma-separated paths defining the contents of the "shared"
# classloader. Prefixes should be used to define what is the repository type.
# Path may be relative to the CATALINA_BASE path or absolute. If left as blank,
# the "common" loader will be used as Catalina's "shared" loader.
# Examples:
#     "foo": Add this folder as a class repository
#     "foo/*.jar": Add all the JARs of the specified folder as class
#                  repositories
#     "foo/bar.jar": Add bar.jar as a class repository
# Please note that for single jars, e.g. bar.jar, you need the URL form
# starting with file:.
shared.loader=${catalina.home}/shared/lib,${catalina.home}/shared/lib/*.jar

#
# String cache configuration.
tomcat.util.buf.StringCache.byte.enabled=true
#tomcat.util.buf.StringCache.char.enabled=true
#tomcat.util.buf.StringCache.trainThreshold=500000
#tomcat.util.buf.StringCache.cacheSize=5000

[BT08]

Mine is exactly the same as yours.  I had already folowed the steps in the KB you provided earlier. DSNs were fine too. :smileycry:

[BT08]

I found another KB about this error but this does not apply to me. Just wanted to post it in case it might be usefull to others:

Storage View, Hardware View and vCenter Service Status not working when using custom SSL certificates:

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=101899...

[BT08]

I found another location where the config references: <soap:address location="http://localhost/sms/services/smService" />

(C:\Program Files\VMware\Infrastructure\tomcat\webapps\sms\WEB-INF\services\smService\META-INF\smService.wsdl)

Can you share yours please?

Thank you.

[alefestaedist]

Sorry for the delay, just checked and it's the same, though can you share me (in private eventually) your entire tomcat folder? I may try to place it on my lab server and see if I can reproduce the error.

Alex

<?xml version="1.0" encoding="UTF-8" ?>
<!--
   Copyright 2005-2010 VMware, Inc.  All rights reserved.
-->
<definitions targetNamespace="urn:sm1Service"
   xmlns="http://schemas.xmlsoap.org/wsdl/"
   xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/"
   xmlns:interface="urn:sm1"
>
   <import location="sm.wsdl" namespace="urn:sm1" />
   <service name="smService">
      <port binding="interface:smBinding" name="smPort">
         <soap:address location="http://localhost/sms/services/smService" />
      </port>
   </service>
</definitions>

[Generious]

Good Morning,

I'm seeing the exact same issue did a upgrade from 4.1 to 4.1 Update last night getting the exact same error message when trying to view the license details...

The upgrade itself was fine no error message and everything else apart from this and the "Storage Management Service" is stuck on "service initializing"

restarting the web service service still the same problem.

No additional applications apart from the vCenter/converter/update manager are installed, IIS isn't installed either.

In the release notes this was meant to have been fixed.

http://www.vmware.com/support/vsphere4/doc/vsp_vc41_u1_rel_notes.html

"

Licensing

• Cannot view the licensing usage reports with the Licensing Reporting Manager
  When you access the Licensing Reporting Manager to view the licensing usage reports in vCenter Server, the vSphere Client might display an error similar to the following:
  Authentication Required
  
  This issue is resolved in this release. "
  

[Generious]

just after checking the SMS.log I can see the following in the log.

faultCode: ServerFaultCode
faultsubcode:
faultString: The session is not authenticated
faultactor:
faultNode:
faukltDetail:
(urn:internalvm25)NotAuthenticatedFault:<object serverGuid="95109C80-9131-4ED2-A925-B09FFDED0628" type="Folder">group-
d1</object><privilegeID>system.View</privilegeID>

The session is not authenticated.

So this tells me the URL end is fine but I might have a problem with the DB account.

creating a new domain service account on the DB and reconfiguring the webservice to use this account instead.

(It was using local system before)

[alefestaedist]

I would follow this article to be sure that you do not have problem with the sql login account.

http://kb.vmware.com/selfservice/microsites/search.do?cmd=displayKC&docType=kc&externalId=1012498&sl... 0 160736159

About the authentication error, do you have any warnign/error in windows log?

Alex

[Generious]

Hi,

I have just checked the KB and verified the account and event logs on the VC host and the SQL Cluster node all appear to be set correct and not seeing any locked out/ general account/denied issues

I have also created a new dbowner account and reconfigured odbc and the webservice to use this account still getting the same problem.

So the account side is okay from what I can see.

Kevin