I’ve been looking into vSphere 5.1’s Essentials Plus free Virtual Storage appliance (VSA). So first a few gotchas – first it requires four NICs before it’ll install, and it seems to take over all spare storage, leaving no local storage for VMs which you don’t what to run with HA – e.g. if you have two DNS servers – HA is irrelevant, as you have fault tolerance at the Application layer.
Haven’t tried this, but I suspect you’d just need to divide your local storage into two chunks – I can do this using my HP Smart Array Controller, so the single disk set presents as multiple virtual disks – one for VSA and one for normal use. Then you install ESXi as normal, create a VMFS partition on the VSA disk before installing the VSA. Once the install is completed, you should be able to put VMFS on the “normal” disk.
I’ve also taken a brief look inside the VSA – thinking it was likely a DRDB based solution, but it actually seems to be is a SUSE 11 VM using MD on top of iSCSI to provide the mirrored volumes. So if you’ve got some linux skills then creating a homebrew version which you have more control over is going to be fairly easy.