VMware grid: VM across physical machines

[DJSL]

What are the challenges VMware face in making a VM span multiple physical machines. This would simplify grid technology to the app layer obviously.

If i can see this VMware people must have thought of this. What are the challenges in implementing such a solution?

[DJSL]

a blog post i found on the topic

http://chucksblog.typepad.com/chucks_blog/2006/12/grid_and_virtua.html

[letoatrads]

What I've heard the primary constraint is the network speed to keep Virtualized memory synchronized between servers. As Inifinband and 10G Ethernet becomes more mainline I would expect you would see VMWare making in-roads to this. I was told by a VMWare SE the next major release might see a 'true cluster' with a VM running on 2 physical hosts and the memory kept in sync via an Infiniband connection between servers allowing for true clustering.

[DJSL]

hmmm.. there are many systems out there that keep memory sync'd over a dedicated ethernet conection/network. obvioulsy is still not a simple task.

[Texiwill]

Hello,

No it is not. Digital Equipment developed a line of products that could do this culminating in the system named the Marvel based on ES40 Alpha architecture. It was extremely expensive and incredibly fast. So it is possible.

VirtualIron tried to develop something similar based on infiniband but they shifted to a Xen implementation which does not do this.

It is very difficult to do. However, computer clusters do not share memory as much as they share small chunks of memory using very very fast filesystems, etc. Each compute node is given a discrete problem to solve use either MPI or PVM interfaces. Which would pass the constraints and memory of the problem to each node. Why you envision surpasses this level of communication.

Best regards,

Edward L. Haletky

VMware Communities User Moderator

====

Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education.

CIO Virtualization Blog: http://www.cio.com/blog/index/topic/168354

As well as the Virtualization Wiki at http://www.astroarch.com/wiki/index.php/Virtualization

[mcowger]

Its expected we'll be seeing something like this fromt he hardware virtualization providers like Xsigo or 3Leaf within a couple years.

--Matt

[DJSL]

this is why i've tried to talk about grids rather than compte clusters (although i did add a tag called clustering). The idea of a compte cluster is more loosely coupled and can be very distributed as i'm guessing you know already :).

The way I see it ( with no knowlege of how vmwares software architecture) by having virtualised a processor cycle, memory block etc... half the work is already done. This is the MPI.

[Texiwill]

Hello,

Current Grid computing also works like the compute cluster, you give each machine of the grid a discrete problem to solve. SETI at home is an example of Grid computing. I think Grid is a misnomer more than anything. What you envision has been solved. Digital developed something called the QBB which in effect was a computer within a computer. It was possible to have an OS span multiple QBBs and thereby get more resources for its processes. Granted this could NOT be done on the fly. However, in order to make the QBB concept work and the Marvel is the ultimate example of this (BTW my mistake it was the ES45 based Alphas), the memory backplane interconnect was incredibly complex. It was a massive NUMA based architecture with a nearly 1:1 mapping between all QBBs. It was incredibly slick.

This level of integration may be possible with infiniband, but it would also push the limits of infiniband. If VMware implemented a vNUMA (virtualized numa) based memory architecture internal to the VM (i.e. memory and CPU tied together somehow) then this may be possible. I know that was virtual iron's original premise, but it is a VERY difficult problem to solve.

Best regards,

Edward L. Haletky

VMware Communities User Moderator

====

Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education.

CIO Virtualization Blog: http://www.cio.com/blog/index/topic/168354

As well as the Virtualization Wiki at http://www.astroarch.com/wiki/index.php/Virtualization