Mac OS X Benchmark: native vs virtualized, part 2

I've been really disappointed by my last benchmark of a virtualized Mac OS X running on top of ESXi with graphics card accessed in passthrough mode. So disappointed in fact that I had to make new tests.
This time, I've decided to ditch the six years old XBench, and to use proper video benchmarking tools: Geeks3D GpuTest, and Cinebench. And guess what? Thats better.
To run those tests, I've had to install OS X 10.8.2 because Geeks3D GpuTest doesn't run on Mac OS X 10.6.8. So I dedicated a SATA HDD on my Mac Pro to a fresh install of 10.8.2, created a VM with it and ran both benchmarks, once from the Mac Pro booted from OS X, once from the OS X VM.

In the chart bellow you can find FurMark and GiMark tests results for a native OS X system running on the Mac Pro, and for the exact same system running as a VM on top of ESXi hypervisor. No tuning was done, I've used the default settings for every benchmarks.

Geeks3D GpuTest Native VM
FurMark (AvgFPS / Score) 47 / 2845 47 / 2872
GiMark (AvgFPS / Score) 33 / 2000 7 / 446

FurMark scores the same frame rate on VM and on native OS X. But GiMark is not good at all, with a VM score 4.5x lower than reference.

Cinebench's results are quite interesting too:

Cinebench Native VM
MHZ 2800 2663
CBCPUX 5.038354 3.797552
CBOPENGL 32.284100 27.319487

VM results are quite close from reference, but the CPU frequency is reported as 2.663 GHz instead of 2.8 GHz, and the VM has only 4 CPU threads, instead of 8. This explain the CPU performance drop between native and virtualized OS X. The OpenGL score is quite good, showing only a 15.4% drop.
We are very far from the 87% drop on XBench's OpenGL test.

On the left side the native OS X, on the right side the virtualized OS X:
Cinebench results for OS X 10.8.2 native vs virtualized

Mac OS X Benchmark: native vs virtualized

An important thing about my work-in-progress virtualized workstation setup is that I've created the Mac OS X VM using my very own hard drives, hooked as raw devices (RDM: raw device mapping). So I can boot exactly the same OS directly from the hard drive, or from ESXi into a virtual machine. Quite convenient when the time comes to make comparisons. And now, I can boot the VM with ATI Radeon graphics card plugged in passthrough mode thanks to VMware DirectPath I/O and some tweaking.
While it's not enough to make a workstation (still miss a keyboard/mouse in passthrough), it allows some benchmarks. I've ran XBench on the VM and on the same OS booted natively from the hard drive.

The VM is configured with only 4 CPU. the Mac Pro sports a quad core Xeon capable of hyperthreading, so when Mac OS X boots natively it sees 8 CPU. It might explain the 50% difference on the Thread test, but that will require further testing.

The final result is not good at all. I understand very well that virtualization has a performance cost, but if I want a powerful virtualized workstation I need a setup that will waste as few resources as possible.
Quartz Graphics and User Interface tests show that "desktop" graphics are well supported, but the OpenGL test results are horrendous. With a performance loss of 87%, it predicts much trouble with games. According to this very simple benchmark, the VMware passthrough mode for graphics card seems to be very bad compared to what can be seen on XEN for example.
To be honest, having my hard disks accessed directly via RDM, I though I would have a 10-15% penalty. The 46% drop for sequential access surprises me. As for the GPU, the OpenGL results are so bad I'm wondering if the graphics card is properly passed through. May be some features are just dropped in the process. By the way, the virtualized Mac OS X won't load the screen color profile. May be it's related to the pseudo-VGA screen attached to the VSphere console. Unfortunately I can't get rid of this pseudo-VGA screen yet. Until I find a way to pass keyboard and mouse through to the VM, I need the VSphere console.

Results 259,39 127,18 -50,97 %
System Info
Xbench Version 1,3 1,3
System Version 10,6,8 (10K549) 10,6,8 (10K549)
Physical RAM 24576 MB 12288 MB
Model MacPro5,1 VMware7,1
Drive Type WDC WD1001FALS WDC WD1001FALS (ATA)
CPU Test 205,42 200,27 -2,51 %
GCD Loop 314,75 16,59 Mops/s 305,66 16,11 Mops/s -2,89 %
Floating Point Basic 182,81 4,34 Gflop/s 177,44 4,22 Gflop/s -2,94 %
vecLib FFT 121,5 4,01 Gflop/s 119,14 3,93 Gflop/s -1,94 %
Floating Point Library 385,38 67,11 Mops/s 374,17 65,15 Mops/s -2,91 %
Thread Test 954,74 477,33 -50,00 %
Computation, 4 thr. 989,65 20,05 Mops/s 517,69 10,49 Mops/s -47,69 %
Lock Contention, 4 thr. 922,22 39,67 Mlocks/s 442,8 19,05 Mlocks/s -51,99 %
Memory Test 452,72 370,19 -18,23 %
System 493,4 452,74 -8,24 %
Allocate 746,78 2,74 Malloc/s 877,63 3,22 Malloc/s 17,52 %
Fill 352,03 17116,62 MB/s 287,47 13977,29 MB/s -18,34 %
Copy 526,18 10867,96 MB/s 497,95 10285,00 MB/s -5,37 %
Stream 418,24 313,1 -25,14 %
Copy 422,51 8726,77 MB/s 321,23 6634,92 MB/s -23,97 %
Scale 395,84 8178,02 MB/s 303,88 6278,02 MB/s -23,23 %
Add 438,89 9349,24 MB/s 328,71 7002,18 MB/s -25,10 %
Triad 417,99 8941,89 MB/s 300,37 6425,59 MB/s -28,14 %
Quartz Graphics Test 315,19 300,47 -4,67 %
Line [50% α] 239,24 15,93 Klines/s 232,29 15,47 Klines/s -2,91 %
Rectangle [50% α] 314,61 93,93 Krects/s 296,25 88,45 Krects/s -5,84 %
Circle [50% α] 264,41 21,55 Kcircles/s 251,89 20,53 Kcircles/s -4,74 %
Bezier [50% α] 279,29 7,04 Kbeziers/s 263,55 6,65 Kbeziers/s -5,64 %
Text 875,44 54,76 Kchars/s 836,28 52,31 Kchars/s -4,47 %
OpenGL Graphics Test 306,01 39,01 -87,25 %
Spinning Squares 306,01 388,19 frames/s 39,01 49,49 frames/s -87,25 %
User Interface Test 463,72 405,19 -12,62 %
Elements 463,72 2,13 Krefresh/s 405,19 1,86 Krefresh/s -12,62 %
Disk Test 97,42 72,35 -25,73 %
Sequential 176,21 94,27 -46,50 %
Uncached Write [4K blk.] 180,38 110,75 MB/s 167,14 102,62 MB/s -7,34 %
Uncached Write [256K blk.] 177,43 100,39 MB/s 80,84 45,74 MB/s -54,44 %
Uncached Read [4K blk.] 149,41 43,73 MB/s 51,88 15,18 MB/s -65,28 %
Uncached Read [256K blk.] 207,16 104,12 MB/s 208,09 104,59 MB/s 0,45 %
Random 67,32 58,7 -12,80 %
Uncached Write [4K blk.] 21,3 2,25 MB/s 19,86 2,10 MB/s -6,76 %
Uncached Write [256K blk.] 507,04 162,32 MB/s 300,75 96,28 MB/s -40,69 %
Uncached Read [4K blk.] 159,73 1,13 MB/s 96,75 0,69 MB/s -39,43 %
Uncached Read [256K blk.] 235,97 43,79 MB/s 242,2 44,94 MB/s 2,64 %

Mac OS X on VMware ESXi: ATI Radeon passthrough

Lately I've been quite involved into a virtualization project: running Mac OS X and Windows as workstations on top of VMware "bare-metal" hypervisor ESXi on my Mac Pro. It requires a good knowledge of virtualization and VMware products like ESXi and VSphere, serious sysadmin skills, and lots of perseverance.

I've finally managed to boot a Mac OS X 10.6.8 virtual machine on top of ESXi, on my Mac Pro with a proper ammount of RAM (12 GB), and the graphics card in passthrough mode. That required a manual tweak of the vmx file.
The VM wouldn't boot when configured with both the graphics card in passthrough and more than 2 GB RAM. I've had to add into the vmx file those two lines:

pciHole.start = "1200"
pciHole.end = "2200"

Then I was able to boot my Mac OS X VM with 12 GB RAM and the graphics card in passthrough. Great. I'm still lacking passthrough for USB keyboard and mouse, meaning I need a remote computer with VSphere Client to control my VM using the embedded console. But the VM uses the physical ATI Radeon, and the physical screen, and in theory it could use full GPU power.

It looks like things are working OK, but it'll take time and many more tests to make sure everything is really working. For example, I was not able to launch 3D FPS games like Left 4 Dead and Left 4 Dead 2 into the VM. The game would crash on launch.

Fix a stuck Steam client on Mac OS X

From time to time, the startup of my Steam client on Mac OS X (10.6.8) is incredibly slow. And sometimes, it won't even launch successfully, getting stuck with a Beach Ball of Death.
A quick diagnostic comes from the powerful utility dtruss:

$ sudo dtruss -p <PID of steam process>
__semwait_signal(0x14D03, 0x4D03, 0x1)		 = -1 Err#60
__semwait_signal(0x17C03, 0x3F03, 0x1)		 = -1 Err#60
__semwait_signal(0xC03, 0x0, 0x1)		 = -1 Err#60
semop(0x2000F, 0xB5464C98, 0x1)		 = -1 Err#35
__semwait_signal(0xC03, 0x0, 0x1)		 = -1 Err#60
__semwait_signal(0x4D03, 0x14D03, 0x1)		 = -1 Err#60

If you read a LOT of errors on __semwait_signal and semop lines, you can fix your client quite easily. I must say, it might have some side effects, but I've never seen any.
First, kill the Steam client (right-click on it's icon in the Dock, choose "Force Quit"), then list semaphores:

$ ipcs -s
IPC status from <running system> as of Fri Nov 30 21:28:29 CET 2012
T     ID     KEY        MODE       OWNER    GROUP
s 131072 0xe93c17d9 --ra-------   patpro   patpro
s 131073 0xc0ec4f17 --ra-ra-ra-   patpro   patpro
s 196610 0xb9e1e4e1 --ra-ra-ra-   patpro   patpro
s 131075 0x697a55e6 --ra-ra-ra-   patpro   patpro
s 131076 0x2e726ce1 --ra-ra-ra-   patpro   patpro
s 196613 0xa9ae61d6 --ra-ra-ra-   patpro   patpro
s 131078 0x1a661f70 --ra-------   patpro   patpro
s 196615 0x36dbd757 --ra-------   patpro   patpro
s 196616 0x44433b26 --ra-ra-ra-   patpro   patpro
s 196617 0x3cea9ea0 --ra-ra-ra-   patpro   patpro
s 196618 0xec712fa7 --ra-ra-ra-   patpro   patpro

If your steam client is not running and you read a full list of semaphores, you might want to remove them:

$ for SEM in $(ipcs -s | awk '/^s / {print $2}'); do ipcrm -s $SEM; done

Then, your Steam client should launch faster (well, at a normal speed), and it shouldn't get stuck.
Use at your own risks.

Escaping the Apple ecosystem: part 0

Years from now, I've understood that the evolution of the Apple ecosystem would eventually become an ethic and practical problem for me and for other longtime Mac OS X power users. I will not give details here, but it's closely related to constant patent fights, proprietary appstores and their underlying business model, lack of professional hardware and software, lack of openness, iOS convergence, etc.
The fact is, I'm deeply addicted to Mac OS X (10.6 branch). It sports great functionalities, from great APIs, and is very nice to use (unless you are a linux control freak upset by the quite limited Window Manager of Mac OS X).
From my standpoint it has critical functionalities that I would miss a lot on other OSes, some of them I use more than ten times a day. For example, the ⌘-k combo that allows me to connect to WebDAV, AFP, NFS, and CIFS shares, to VNC servers… So convenient, so irreplaceable. I could find dozens different little (or big) things that make me stick with Mac OS X. Some of them are quite huge: it's not Windows, it runs my software (Adobe's CS5, Valve's L4D2, my beloved text editor BBEdit and many more), it's UNIX under the hood - and I must admit, the hood is constantly open.
I do understand of course that Apple is right about consumer products, they have a very good business model, and the recent rumor about a switch to ARM's CPU makes so much sense. But I'm no regular consumer. I spend 10 to 18 hours a day in front of various computers, have neither smart phone nor facebook account.
Unless I'm ready to give up much control to Apple, there's no way I move to iOS OSX 10.8 / 10.9. That's why I'm studying a path to escape the Apple ecosystem. I want it to be a slow process, allowing me to progressively switch from Mac OS X to something else.

Step zero

I think about step 0 as the core of my project, as the main idea. And that would be to keep on using Mac OS X. Huge step, uh? No kidding.
I'm currently evaluating various solutions that would allow me to:

  • keep Mac OS X as my main OS
  • use Windows, FreeBSD, linuxes as alternative OSes at the same time
  • enjoy full hardware power (use full GPU power, not emulated, not virtualized)
  • never reboot (I'm used to 15-50 days uptime, rebooting to change OSes is not an option)

Hence, I would be able to slowly switch my habits from Mac OS X to other OSes. Some softwares I need (Adobe's for example) will stay on Mac OS X until I buy a new version running on another OS, or until I find a nice alternative. That will take time.

In theory my step 0 would require at least:

  • A bare metal hypervisor running on my Mac Pro
  • a Mac OS X virtual machine capable of using natively USB ports, GPU, and SATA
  • a Windows virtual machine capable of using natively USB ports and GPU
  • optional: a linux VM capable of using natively USB ports and GPU

I've got plenty of power (2.8 GHz quad core Xeon), and plenty of RAM (24 GB). But believe it or not, that's not enough. For example, it's not possible to share the GPU between two active virtual machines. If you want direct I/O ("passthru" mode) for your graphics card, you need one card per active VM. That's not a real problem, My Mac Pro has few empty PCI slots, I can buy a fanless ATI Radeon or two as dedicated graphics cards for VMs.

As far as I know, VMware ESXi 5.1 is the only bare metal hypervisor supporting Mac Pro model 5,1 (mine). And VMware is also the only one supporting Mac OS X 10.6 (server) virtual machine on top of bare metal hypervisor (on top of Apple hardware). Fine.
VMware has "DirectPath I/O" that allows direct bind of a PCI device into a VM. For example, you can create a virtual machine and make it use the PCI GPU directly, so that you would have proper video power inside your VM.
Unfortunately this passthrough mode has severe limitations. Biggest ones are:

  • You can no longer create snapshot of your VM
  • You can no longer suspend your VM

I've installed VMware ESXi 5.1 on a spare SATA hdd into my Mac Pro. I've created a Windows virtual machine, and hooked the PCI graphics card on this VM using DirectPath I/O. It worked great. As soon as I've installed ATI drivers, the display plugged on the Mac Pro was reclaimed by the Windows VM. I was not able to dedicate USB ports to the VM, so for now it's kind of useless. Trying to launch a Valve game (Half Life Lost Coast) eventually fails. Probably a software configuration problem.
The same setup for a Mac OS X VM won't boot. I'm able to boot a Mac OS X virtual machine, using my very own SATA hdds in raw device mapping (ie. it boots my regular Mac Pro OS, from its physical hdds, as a VM). But if I try to use the ATI card via DirectPath I/O, it won't boot.

I'm stuck at step zero, with important questions asked, and not answered:

Is it possible to use GPU card passthrough with a Mac OS X guest? (also here)
Is it possible to configure USB passthrough on a Mac Pro with ESXi 5.1?

Feel free to help (and to correct my english). I really need those issues to be solved in order to move forward with this project.

helpful links:
fan speed of the ATI card with ESXi 5.1
Raw Device Mapping of local SATA disks on ESXi
ESXi as a Desktop with VMDirectPath I/O

Utiliser auditd sur Mac OS X et FreeBSD – 3

Cet article est la suite de Utiliser auditd sur Mac OS X et FreeBSD - 2

Exploiter les résultats de l'audit

Dans le premier article j'ai mentionné très succinctement une commande qui permet de lire les logs d'audit en temps réel. Et finalement la consultation de ces logs est sans doute la chose la plus intéressante. Personne n'a envie d'activer l'audit sur une machine pour le simple plaisir de remplir son disque dur. Tout ceci doit avoir une raison d'être : pouvoir ensuite (ou en temps réel) exploiter les logs d'auditd. Continue reading

Utiliser auditd sur Mac OS X et FreeBSD – 2

Cet article est la suite de Utiliser auditd sur Mac OS X et FreeBSD - 1

Configuration : quid des utilisateurs comme www ?

J'expliquais précédemment que le système d'audit ne peut suivre un utilisateur que si ce dernier se connecte à la machine (par ssh par exemple), et que quelques soient les ruses utilisées (su, sudo...) c'est toujours l'UID réelle de l'utilisateur qui est suivie.

À cause de cela, il est totalement impossible de traquer des évènements appartenant à des utilisateurs qui ne peuvent pas se connecter au système. Ainsi, une ligne comme celle-ci dans audit_user ne permettra de traquer aucun évènement :

# pour freebsd remplacer _www par www

Il serait pourtant très intéressant de savoir ce que fait Apache dans votre dos, parfois sous la direction de méchants pirates.
De nos jours, La plupart des serveurs utilisent des protocoles comme l'HTTP qui permettent aux utilisateurs d'accéder aux ressources sans être authentifiés au niveau du système. Les utilisateurs en question n'existent d'ailleurs pas au niveau du système. Néanmoins, les applications web permettent de faire énormément de choses via les langages comme le PHP, y compris d'interagir avec le système, en lançant des commandes, en créant des fichiers, etc.

Si on souhaite qu'auditd puisse contrôler les actions de www, il faut recourir à un artifice. Attention tout de même, ce qui suit est plus proche d'un hack que d'une méthode vraiment propre, et je ne peux pas garantir son fonctionnement correct dans un environnement de production.

En premier lieu il faut installer un programme spécifique, qui va permettre de forcer auditd à suivre les actions pour une UID donnée, sans que l'utilisateur correspondant ait besoin de se connecter à la machine.

curl -LO
cc -o setaudit -lbsm setaudit.c 

Cela vous donne, si tout se passe bien, le binaire setaudit, que l'on va utiliser sous cette forme :

setaudit -a UID_WWW -m FLAGS /programme/de/lancement/d/apache

Pour suivre les évènements d'exécution de commande sous respectivement FreeBSD et Mac OS X, cela donnera :

setaudit -a www -m ex /usr/local/etc/rc.d/apache22 restart
setaudit -a _www -m ex /usr/sbin/apachectl restart

Ainsi, toutes les commandes lancées par Apache (donc sous l'UID www) sont enregistrées par auditd.
Comme le masque des évènements (les flags de l'argument -m) décrit ce que l'on doit suivre, il est même inutile de renseigner le fichier audit_user.

Pour résumer :

  • Si vous souhaitez que l'audit suive les utilisateurs qui se connectent à la machine, vous pouvez régler les classes d'évènements qui seront soumises à l'audit pour l'ensemble des utilisateurs dans le fichier audit_control, et gérer les cas particuliers dans audit_user.
  • Si vous souhaitez que l'audit suive des UID "interne", pour traquer tout signe de compromission sur un serveur web, un serveur de mail, etc. il faudra recourir à setaudit pour activer l'audit au moment du lancement du service correspondant.
  • Faites attention à la liste de flags que vous choisissez. Le volume des logs d'audit peut grossir très rapidement.

Troisième partie de l'article ->