VMWareTasks 1.3 for VixCOM 1.7.0 (let's reinvent less wheels)

[dblock]

I've posted a release candidate for VMWareTasks 1.3 that supports VixCOM 1.7.0. Please download it here and file bugs or post questions at .

Features

- Added a VMWareComLib and VMWareComTools that expose a full scriptable COM interface.

- Added a vestris-vmwarecomlib.jar and vestris-vmwarecomtools.jar that exposes a Com4J JNI wrapper for VMWareComLib and VMWareComTools to Java clients.

- Added support for VixCOM 1.7.0. VMWareTasks works against both 1.6.2 and 1.7.0.

Misc Improvements

- All VMWareLib.Tools objects are now IDisposable and explicitly release reference to the virtual machine.

- All assemblies are strongly named and signed.

- The source in the release package can now be built without any changes.

Bugs

- VMWareRootSnapshotCollection.CreateSnapshot doesn't pass flags to the COM API.

Backwards incompatible interface changes

- VMWareVirtualHost.ConnectToVMWareServer takes an additional username and password. Pass blank values to connect to a local VMWare Server 1.x.

I am working on more thorough multithreaded stress tests, but with some early testing it looks like the VMWare development team did a very good job at fixing all the issues that I have reported, thank you!

[fixitchris]

I see that IVixHandle supports a new Close() method.

How does this work in relation to multiple threads?

[dblock]

In order to work in a threaded scenario all handles must be closed after they aren't used any more.

All handles have become IVixHandle2-s. All except IHost need to be Close'd. IHost just needs to be Disconnect'ed. For those who haven't switched to VMWareTasks yet, this is going to be quite a job, since all handles, including for example jobs need to be closed this way to work in a multi-threaded scenario. If you are using VMWareTasks, you don't have to do anything except upgrading to the new version - all handles are wrapped by VMWareHandle, so I had to make two lines of code changes.

[admin]

fixitchris: Once you have called Close on an object, you should not call any other operations for that object. So if you want to use the same object in multiple threads, you must ensure Close is called only after all threads are done with the object. If you have any other questions, let me know.

[fixitchris]

by Object do you mean a host or do you mean an instance of IVixHandle2 to an IHost interface inside my code?

[fixitchris]

In other words can I have multiple handles to the same host across my threads ? If I can then calling Close on any one of those open handles will invalidate all other open handles to that same host? Or can I have only one handle to a single host at any time, but 1.7 allows me to connect to multiple hosts at once?

[admin]

fixitchris: Close is not used with the host object/handle. Close is only used with VM, Job, and Snapshot objects. If you copy any of those objects for use in a multi-threaded scenario, none of those copies (or the original) should be used once Close has been called on any of the copies (or the origina)l. Check the updated Vix documentation for Close under COM Functions. It provides more details and sample code which may help to clarify things.

[fixitchris]

jk: thanks for clarifying. Reading through the new docs I'm seeing that Disconnecting a host while another thread is executing a job.Wait could present a problem.

dblock: VMWareVirtualHost inherits from VMWareVixHandle, which implements IVixHandle2.Close on Dispose. JK mentioned that Close is not used with the host handle. Am I missing something?

[dblock]

The code avoids closing Close in a VMWareVirtualHost.

public override void Dispose()

{

if (_handle != null)

{

Disconnect();

}

GC.SuppressFinalize(this);

}

public void Disconnect()

{

if (_handle == null)

{

throw new InvalidOperationException("No connection established");

}

_handle.Disconnect();

_handle = null;

_serviceProviderType = ServiceProviderType.None;

}

[fixitchris]

dblock: I think I managed to crash the VIX lib by closing a host handle while executing job.wait. ( I have to track it down some more ) Have you run into any crashes yet?

[dblock]

Yes, I've seen that happen. The API doesn't support disconnecting while waiting on jobs - you need to dispose of the job first. Given it's a C API at the core I think it's acceptable that it AV-s if you do things you shouldn't be doing with it.

[fixitchris]

It looks like it doesn't support Connecting while waiting for a job to complete.

Created Thread1 and Thread2

Thread1 creates a connection to Host1

Thread1 is waiting for a job to complete

Thread2 creates a connection to Host1

...exception.

[dblock]

Update: VMWareTasks 1.3 has been released > download.

[fixitchris]

dblock, why the necessity for GC.SuppressFinalize in VMWareVirtualHost?

[dblock]

Cause it has been disposed. It avoids wasting the finalizer's time.

[fixitchris]

http://communities.vmware.com/thread/187177?tstart=0

I ran into an issue where I upgraded my project to vmwaretasks 1.2 from 1.1 on VIX 1.6.2 and I started getting VIX exceptions usually between host connect/disconnects on a single thread. I forced GC.Collect and the issues went away. I wonder why this was not a problem with vmwaretasks 1.1.

[dblock]

I've been seeing these problems against ESX4. I ended up nailing down every single issue: objects like Snapshots are holding a handle and hence need to be closed. It gets rather complicated because a lot of things refer to each other in VMWareTasks, but I think I cleaned that all up.

Calling GC.Collect() effectively does dispose the unreferenced objects, so you're okay. I did make snapshots and collections of snapshots IDisposable for 1.4 (download a daily build) so you can use the using construct or calling Dispose (or in 1.3 you can call Close) explicitly. If you do this consistently throughout your code, you will not need to call GC.Collect(). If you get an AV with a daily 1.4 build without a GC.Collect, then you forgot to dispose something that you got from VMWareTasks (eg. a VMWareSnapshot) or there's a bug in VMWareTasks. If you think it's the latter, bring it up on the project's discussions.

I also reported one bug to VMWare where CreateSnapshot AVs with a callback.

[fixitchris]

Looking at the VMWARECRASH project, I wonder if the approach needs to be changed to use:

Marshal.AllocHGlobal

Marshal.FreeHGlobal

to allocate and deallocate Ijob/ivm2/isnapshot.

What do you think dblock?

edit: I have to think more about this.

[fixitchris]

This is interesting, in vmwarecrash:

object openResults = null;
rc = openJob.Wait(openProperties, ref openResults);
openJob = null;

but openResults is never explicitly destroyed.

Your code VMWareVixHandle:

public object[] GetProperties(object[] properties)
    {
        object propertiesArray = null;
        VMWareInterop.Check(this._vixhandle.GetProperties(properties, ref propertiesArray));
        return (object[]) propertiesArray;
    }

you're returning propertiesArray , maybe a copy of propertiesArray should be returned instead.

Also, the System.Object usage in Ijob.Wait could be investigated further:

http://msdn.microsoft.com/en-us/library/2x07fbw8(VS.80).aspx

http://blogs.msdn.com/adam_nathan/archive/2003/04/24/56642.aspx

HRESULT
Wait([in] VARIANT propertyIDs,
     [in,out] VARIANT* propertiesArray,
     [out,retval] ULONGLONG* error);

propertiesArray is a pointer.

Also looking at the sample C I see some interesting features that we could possibly p/invoke to our advantage:

VixHandleType Vix_GetHandleType(VixHandle handle);
void Vix_ReleaseHandle(VixHandle handle);

For some reason I don't think that the VIX unmanaged code is at fault with the issues we're having, it's either the COM implementation or something on the CLR side. However I would love to know how multiple handles are managed inside VIX.

[fixitchris]

HRESULT
GetProperties([in] VARIANT propertyIDs,
              [in,out] VARIANT* propertiesArray,
              [out,retval] ULONGLONG* error);

propertiesArray could be traversed if we could keep it as a pointer:

OurCurrentPointer = IntPtr.op_Explicit(OurCurrentPointer.ToInt32 + (Marshal.PtrToOurExpectedDataType(OurCurrentPointer).Length + 1)

expected data type:

Vix_GetPropertyType(VixHandle handle,
                    VixPropertyID propertyID,
                    VixPropertyType *propertyType);

VIX_PROPERTYTYPE_ANY 
Indicates that no property type has been assigned to this variable. Recommended for initializing property type variables. 

VIX_PROPERTYTYPE_INTEGER 
The property type is 'int'. 

VIX_PROPERTYTYPE_STRING 
The property type is 'char *'. 

VIX_PROPERTYTYPE_BOOL 
The property type is Boolean. 

VIX_PROPERTYTYPE_HANDLE 
The property type is VixHandle. 

VIX_PROPERTYTYPE_INT64 
The property type is 'int64'. 

VIX_PROPERTYTYPE_BLOB 
The property type is 'char *". When returned as a job property, the blob is returned as two values: first an 'int' containing the blob size in bytes, then a pointer to the blob.