Lately I've been quite involved into a virtualization project: running Mac OS X and Windows as workstations on top of VMware "bare-metal" hypervisor ESXi on my Mac Pro. It requires a good knowledge of virtualization and VMware products like ESXi and VSphere, serious sysadmin skills, and lots of perseverance.
I've finally managed to boot a Mac OS X 10.6.8 virtual machine on top of ESXi, on my Mac Pro with a proper ammount of RAM (12 GB), and the graphics card in passthrough mode. That required a manual tweak of the vmx file.
The VM wouldn't boot when configured with both the graphics card in passthrough and more than 2 GB RAM. I've had to add into the vmx file those two lines:
pciHole.start = "1200" pciHole.end = "2200"
Then I was able to boot my Mac OS X VM with 12 GB RAM and the graphics card in passthrough. Great. I'm still lacking passthrough for USB keyboard and mouse, meaning I need a remote computer with VSphere Client to control my VM using the embedded console. But the VM uses the physical ATI Radeon, and the physical screen, and in theory it could use full GPU power.
It looks like things are working OK, but it'll take time and many more tests to make sure everything is really working. For example, I was not able to launch 3D FPS games like Left 4 Dead and Left 4 Dead 2 into the VM. The game would crash on launch.
I've been messing around with this since stumbling onto your site. I was looking for more processing power for video and decided to experiment putting Mountain Lion on my whitebox ESXi 5.1 host that has 16 vCPU and 24 GB RAM. I was unable to run Final Cut X as it wanted accelerated graphics, so I tried passing through a Radeon HD6570 PCIe card, and it seems to work now.
The next step is to see if I get audio and everything works in sync.
My writeup is here: http://www.insanelymac.com/forum/topic/285733-graphics-acceleration-in-esxi-51-and-os-x-mountain-lion-vm/
Hi Stephen,
I'm afraid playing with sound is doomed to failure. It's officially unsupported by VMware (they write somewhere in the documentation that passthrough of sound is not functional). May be a dedicated sound card (PCIe) would work.
I've read that some people get proper sound playback using "USB sound card", but it's on PC hardware, running Windows VM on top of ESXi. Not sure it can apply to a "Mac" situation (http://networkingbunny.co.uk/documents/vmdirectpath ).