February 1, 2022

So it's been a little over a year since the last macOS on ESXi install guide I've written for Big Sur and VMware ESXi 7 Update 1. Subsequently, Apple has released macOS 12 Monterey and VMware has updated ESXi to 7 Update 3

More...

Although mostly unchanged from the previous guide, I thought I'd write an updated and more simplistic guide to running macOS on vSphere 7. πŸ˜€

Below you'll find the step-by-step instructions on how I got this installed in my home lab

DISCLAIMER

This should only be run in a lab environment and not in a production environment. Don't be that person! πŸ™„

- Please note: This guide is written for an Intel compute based machine running a Type 1 bare metal hypervisor and not a Type 2 like VMware Workstation or VMware Fusion. -

** This guide was written using vSphere 7 U3c build 19193900 & macOS Monterey 12.2 **

Prerequisites

  1. VMware ESXi 7 Update 3
  2. macOS 12 Monterey ISO
  3. macOS Unlocker

First we'll need the above 3 items in order to complete this whole process. Relax! I'll hold your hand through all the steps in obtaining these prerequisites. 

VMware ESXi 7 Update 3

1. Download VMware ESXi 7 Update 3

So first things first! We gotta grab the latest ESXi version and install it prior to installing macOS 12. Go to the downloads page here and select the version of ESXi you have and want to install/update. 

2. Install/Upgrade VMware ESXi 7 Update 3

Once you've selected the correct install for your licensed version of vSphere, you'll simply need to install it. I've written a previous guide on how to Upgrade from an offline bundle here. This is the path I went with in order to update the current ESXi install in my lab. 

macOS 12 Monterey ISO

Next we'll need to obtain the macOS 12 Monterey ISO. I'm sure it'll be easy to search the net and find the ISO, but ensure you get it from a valid trusted source. Otherwise make sure you validate the SHA-1, CRC and MD5 values of the image to make sure it hasn't been tampered with before installing it. πŸ™‚

I'll take the safer route and create it myself using a macOS native machine. If you have access to a MacBook, iMac or Mac mini already on Monterey, you should be able to create an ISO from any one of those machines.  

Can't you hook a brotha up and let me direct download the ISO from your site? 

1. Download macOS 12 Monterey Installer

  • Once you have a macOS native machine running Monterey in front of you, navigate to the App Store here to download the obtain the Monterey installer. Once you click the link, you should see a browser pop-up asking you to open App Store.app
  • From there, the App Store app should open up to the Monterey page. click on the Get button to download Monterey. 

Once you click on Get you'll be greeted with a download prompt. Click on Download then wait for the download to complete. Download times will vary based on your network speed. 

When the download is finished, you'll be prompted with the install window to Continue. However...

DO NOT CLICK CONTINUE!

Instead, simply press Command + Q to at this install window. Once you do, simply click on Quit at the prompt to close the window.  

If you did everything correctly, you should now have the Install macOS Monterey.app listed in your Application Folder. This app is what we'll use to create our ISO in the next step. 

2. Mount Installer then Convert to ISO

Now that we have the Monterey App, we'll run a few commands to first mount the installer then convert it to an ISO with the purpose of mounting it to a VM in ESXi. 

  • Launch the terminal and run the following command. 
sudo -i

Running this command will prompt for the administrator password. Once you type it in, you'll now be logged in as the root user which will be the account we'll use to execute the rest of the commands. 

  • From the root# prompt, enter in the following command:
 hdiutil create -o /tmp/monterey -size 13800.1m -volname monterey -layout SPUD -fs HFS+J

This will create the directory /tmp/monterey.dmg

** This 13800.1m size for this guide was based on the version 12.2 of macOS and may be different for any updates/patches on future 12.x versions. **

  • After this we'll need to mount the dmg using the following command:
hdiutil attach /tmp/monterey.dmg -noverify -mountpoint /Volumes/monterey
  • You should see the following output below:

You should now also be able to see the mounted disk under Disk Utility. 

  • Next we'll create the install media from our new mount with this command:
sudo /Applications/Install\ macOS\ Monterey.app/Contents/Resources/createinstallmedia --volume /Volumes/monterey --nointeraction

You should see the following result below while making the disk bootable. 

  • Afterwards, you can eject the attached install disk using the following command:
hdiutil eject -force /volumes/Install\ macOS\ Monterey

You should see that the install disk is ejected now. 

  • Next we'll need to convert the image to a .cdr with the following command:
hdiutil convert /tmp/monterey.dmg -format UDTO -o /Users/erickdimalanta/Desktop/monterey.cdr

Don't forget to change the path /Users/erickdimalanta/Desktop/monterey.cdr to your own specific path you are working out of. Once you run the command, you should now see a monterey.cdr file in your working directory that is about 14.47 GB. 

Now that we have our media, we simply need to rename the file extension to a .iso. You can do so with the command below by running an in-place mv(move) command. 

mv /Users/erickdimalanta/Desktop/monterey.cdr /Users/erickdimalanta/Desktop/monterey.iso

Again, make sure you set "YOUR" correct path if you are copying and pasting the command above. 

3. Cleaning Up

Finally let's clean up after ourselves since leaving remnants of leftover data is just icky! πŸ˜›

We are going to clean up the initial directory where we created the monterey.dmg in the following path /tmp/monterey.dmg. Use the below command to cleanup. 

rm -rf /tmp/mont*
This above command will make sure to force delete everything in the specified directory without prompting for confirmation. So basically we are removing everything in the /tmp folder that starts with "mont." Make sure nothing else starts with "mont" otherwise you'll remove that also. 


The below screenshot shows how to simply validate and double check afterwards. 

4. Upload Monterey ISO to Datastore

For the last step in this section, we'll need to transfer the newly created Monterey.iso to our VMware environment. You can upload it to a datastore or Content Library if you are also running a vCenter. To keep it simple, I'll showcase uploading it to a datastore. 

  1. Login to your ESXi host
  2. Navigate to Storage and select the Datastore you want to place the .iso in. 
  3. Click on Datastore Browser
  4. Select the Datastore 
  5. Select the folder you'll place the .iso in. I'll use the ISO folder
  6. Then click on Upload

Navigate to the directory where you created the monterey.iso. Select the monterey.iso and click on Open

Review the Recent Tasks pane, do a run around the block, then monitor to completion. 

macOS Unlocker Installation

Once the .iso is finally uploaded to the datastore, it's now time to prep our ESXi host with the ability to run macOS in a VM. The unlocker is written python that modifies the vmware-vmx file to allow macOS to boot. Without this unlocker, the machine simply doesn't work and is suck in a sad infinite boot loop. 

1. Download the Unlocker

You'll first need to download the unlocker zip file here

The above link should be a direct download to the unlocker from github. The file name is esxi-unlocker-master.zip

2. Upload to Datastore

Once you've downloaded the esxi-unlocker-master.zip, you'll need to upload this to the same location we uploaded the monterey.iso in a previous step above. 

3. SSH into ESXi host to Install the Unlocker

Bring up your terminal if you've previously closed it and SSH into your host.

- Make sure you enable SSH on your host if disabled. -

From there, login to your ESXi host then navigate to your unlocker location as shown by the screenshot below. 

  • Once in the parent location of the unlocker, we'll need to unzip the esxi-unlocker-master.zip file. You can do so with the command below:
unzip esxi-unlocker-master.zip

The below image should be your resulted output.

4. Set permissions on the extracted folder

Before we navigate to the newly extracted folder and install the patch. We'll need to set the permissions on the folder and files within. Not doing so will result in a "Permission denied" error as shown below. 

So since I'm here to save you the headache so that this install guide works the first time.

  • Enter the following command below:
chmod 775 -R esxi-unlocker-301/

This command will grant permissions to the folder and all the files recursively within. 

5. Install Unlocker Patch

Once you've set the permissions on the folder and files above, we can now install the patch. First navigate to the extracted folder that contains all the files necessary for the patch. 

  • Enter in the following command:
cd esxi-unlocker-301/

Once there, we can actually use a patch validation checker to make sure the patch is installed or not. I know we haven't installed the patch yet, but let's just run that first to see the resulted output.

  • Run the following command to validate:
./esxi-smctest.sh

After running the command, it will obviously show false since we've not yet installed the patch. Well let's do it now! 

  • Run the following command to install the patch:
./esxi-install.sh

Once you've run the install ./esxi-install.sh, we'll need to now restart the server for the configuration changes to take effect. In the same terminal window, you can simply type reboot then press Enter. 

reboot

I know it goes without saying, but please make sure you have no running VMs before typing reboot. πŸ˜›

6. Validate Successful Patch Install

When the ESXi host is back up from it's reboot, let's SSH into the host again and navigate to the same location where we ran the patch installer. 

Once there run the validation command again to now see a different result.  

./esxi-smctest.sh

This should now display smcPresent = true which means we are now ready to install macOS without running into any boot issues. 

macOS 12 Monterey VM Install

Whew! It's about time amirite? 

If you've followed the above steps correctly, you should now be able to continue on and install Monterey!

1. Create a New Virtual Machine

Go back to your ESXi host and navigate to the Virtual Machine menu item on the left. Then click on Create / Register VM

Another screen will pop up. From there select Create a new virtual machine and click on Next

Next, type out a name for your new VM. Select the following options from the drop down menu for the rest of the categories.

  • Name: ANY NAME SHORTER THAN 80 CHARACTERS
  • Compatibility: ESXi 7.0 U2 virtual machine (U3 wasn't available despite being on U3)
  • Guest OS Family: Mac OS
  • Guest OS version: Apple macOS 12 (64-bit)

 Finally, click Next when finished. 

From here, select which datastore you'll want to place your machine in. 

The next option will be to configure the hardware specs of the VM. 

You are going to want to set your CPU, Memory, Storage size, type of provisioning, Network, and most importantly configure your CD/DVD Drive 1 to boot from the monterey.iso we've created. This is located in the ISO folder on the datastore we uploaded it to in an earlier step

Click on Next once done. 

Finally, confirm all your settings are what you want, then click on Finish when complete. 

2. Power On VM & Format VM Storage

Sweet! Now that you are done creating your VM, let's power this bad boy on to see if it bypasses the boot loop error since we applied the Unlocker patch

Boom! Success! Boot loop is no more!

Anyway, continuing on with the install. Select your language of choice and click on the bottom right arrow. Next we'll need to format the newly provisioned disk so that macOS can recognize the storage to install itself onto. 

  • Select Disk Utility and click on Continue.
  • Next, highlight the VMware Virtual SATA Hard Drive Media and click on the Erase option at the top.

Enter in a Name to label your disk and keep the defaults for Format - APFS and Scheme - GUID Partition Map.

  • Click the Erase button when ready
  • The formatting process will proceed. Once complete, click Done.
  • We can now quit Disk Utility by navigating to the top menu and clicking on Disk Utility > Quit Disk Utility

3. Install macOS 12 Monterey

  • Now we can install the OS. From here select Install macOS Monterey then click on Continue.
  • On the next screen click Continue

Click Agree twice for the Software License Agreement

Click on the newly formatted HDD then click on Continue.

macOS Monterey will start to install now...

Don't believe the time remaining is even close to accurate. I would go bake a frozen pizza and watch a movie or something, then come back waaaayyyyy later. πŸ˜›

When the never ending string of installation reboots are completed, the install will be complete! Again, I won't go through this self-explanatory setup wizard. 

Install VMware Tools (Optional but necessary)

Finally, the last thing to do after you've gone through the initial OS setup wizard, is to install VMware tools. You can also unmount the monterey.iso from the VM at this point since it's no longer needed.  

Wut the heck are VMware tools? 

1. VMware Tools Error 21001

You'll likely receive the below error when attempting to mount the VMware Tools from the host.

Failed - The required VMware Tools ISO image does not exist or is inaccessible. 

This occurs because the darwin.iso doesn't exist in the directory "/usr/lib/vmware/isoimages." Which is fine as these are the iso images that shipped with this version of ESXi that I installed. According to the KB 2129825 we have a few options in resolving this. I'm going to simply go with the option to download the latest VMware tools for this specific OS.

2. Download VMware Tools

  • Make sure you are logged into your Customer Connect account. Then click Here to Download the latest VMware Tools. I selected the .zip option.

Next, navigate to your download location and extract the .zip. On macOS you can simply double-click the file to extract. 

Sweet! Now you should have the darwin.iso image we need to install VMware Tools onto our macOS 12 Virtual Machine. 

3. Mount VMware Tools ISO to VM

Now that we have the VMware Tools image for macOS, we'll need to upload the .iso to a datastore the same way we Uploaded the monterey.iso in an earlier step. 

Once the iso is uploaded there, Edit settings on the VM and select the darwin.iso to mount just like we did when we were creating the VM. When done, click Save

So...who in the heck is Darwin? 

4. Install VMware Tools in Monterey

Next we'll hop back over to our VM and we can now see that VMware Tools is mounted and ready for install. 

Double click on the Install VMware Tools icon. From there the Tools installer will launch. Click on Continue to proceed. 

Click Install on the next screen

You'll then be prompted to enter in the password you set when you went through the initial OS setup wizard. Once done, click on Install Software. Then click OK on the administer prompt. 

You'll then likely receive a System Extension Blocked prompt during this process. Click on Open Security Preferences. At this point the Security & Privacy window will open up. Once opened, click on the lock in the bottom left and enter in your OS password, then click Unlock

Once done and the lock icon is unlocked, click on the Allow button to allow VMware Tools to continue loading. 

You should now receive a prompt to Restart before using the system extensions. But before doing so, click on the open Installer in the dock below. 

The VMware Tools window should showcase that the installation was successful. Go back to the Security & Privacy window in the background and click on Restart from there.  

Once the machine has rebooted, you should now see that VMware Tools are installed for this VM. From there, you can either eject the mounted VMware Tools image from the OS or disconnect it from the CD/DVD Drive 1 option in the VM under Edit settings

Schweet! All done! You should now have a macOS 12 environment that you can test and break all you want! Have fun! 😎

**Credits to shanyungyang's github and everyone else involved that helped me put this updated simple guide together.**

RELATED POSTS

  • Erik,

    I have used both the Big Sur and this Monterey VM setup guides and they both worked perfectly.

    I am having a similar issue with using the yellow minimize button on windows causing a delay in closing. Brad and Ralph both mentioned this as well. Wondering if there are any updates on this?

    I also had an issue with chrome. I had to turn off the hardware acceleration for it to render correctly.

    I was wondering if you have looked at doing this same type of install on a mac Running VMware Fusion? Is it possible to do that?

  • Ran through your process and got FANTASTIC results…!!!

    I have a 4-core 3.4 GHz proc’d server loaded with macOS 12.3.1 (Monterey) at the time of this posting, running VMWare ESXi v7.0U3. It works be-a-u-tifully…!!!

  • How do you format and where do you insert info such as serial number, product name, bord product, MLB and ROM in the vmx file in order to have working iservices, Apple ID etc?

  • Hi Eric,
    thanks a lot for this great guide.
    Took me ~2 hours to get MacOS running on ESXi 7U2 (and a lot of that was just install / waiting time). Excellent description of steps and really easy to follow.

    2 suggestions:
    – My system runs on an Ryzen 5 3600 and got stuck in the boot loop until I found the solution in the comments. In your old article the AMD notice was included in the article itself at the top, could you do this here as well?
    – Your config screenshot shows 4 CPU / 4 GB RAM, which I used at first. Found the system to be sluggish and I increased to 8 CPU / 8 GB RAM. Not sure if 4/4 was ok for older OS versions, but here it seems a bit low.

    Thanks again for the great work, I will now try to get some applications running on my new Virtual MacBook.

  • You deserve a prize. This tutorial or instruction is very detailed. However I setup ESXi on a power mac and I didn’t have to use unlocker. Can you explain why that is the case? Was it that my version of 7.x is later?

    • Because you are using Apple hardware (assuming you meant that you installed ESXi *on* your power mac). The unlocker lets you bypass Apple’s hardware license agreement, which basically only allows MacOS to run on Apple hardware.

  • Great tutorial! I have downloaded Monterey 12.3. How did you calculate the DMG size as 13800.1m?
    For 12.3 I’m getting 12265.4 and am surprised it is smaller.

  • This may not be the best place to post this question but it’s tough to get community support for this type of installation. Does anyone else have the issue where minimizing any window using the yellow “-” button and then restoring the window by clicking on the app in the dock; either operation causes a temporary freeze (e.g. several second delay)? CPU/GPU/Memory utilization is very low when this happens so I’m having a tough time figuring out what’s causing the issue. Hiding any window using the red “X” button and restoring the hidden window is snappy. This one has me puzzled. I would really appreciate it if anyone has any ideas on how to solve this issue. Thanks!

    • I have the exact same issue. It appears consistently whenever I minimize a window to the Dock. Closing and opening is fast.
      This is apparently caused by the “windowserver” process, which is the MacOS process actually handling all the screen / window actions. When minimizing to the Dock the CPU usages spikes for this process, causing delay. When symply closing the window the process is unchanged.
      Switching off the animation for the docking process changed nothing.
      I suspect this is related to the usage of the VMWare graphics card. There are other issues related to graphics on the system as well (e.g. videos don’t play in Safari but work fine in Firefox).
      I am trying to passthrough a Radeon to this machine, but have not got that working yet.

  • Hey Erick, thanks as always for the wonderful article. Noticed a minor case issue that tripped me up for a few minutes in case you want to fix it for others. The “v” in volumes in the command below should be uppercase “V”.

    hdiutil eject -force /volumes/Install macOS Monterey

    Corrected:
    hdiutil eject -force /Volumes/Install macOS Monterey

    • Hi Brad. Thanks for sharing. That’s strange you had that issue. I just went through the process I listed out in this blog and copied and pasted all the commands without any errors. The hdiutil eject command with a lower case “v” in volumes worked fine for me.

  • Erick,

    Great post and much appreciated the detail put into it. I just installed the latest ESXi version 7.0.3 Build 19193900 which is the exact same as the one described in this article.

    I followed everything to the “T”. I apparently still get the infinite loop with the Apple logo when I try to install (either Monterey or Big Sur). One difference I notice is that there is no progress bar underneath the Apple logo in mine.

    I noticed another difference is that I’m AMD and you’re Intel. But I don’t see that as being an issue since it’s all ESXi, same version numbers, etc.

    I’ve been scouring around the net since yesterday almost non-stop to try and find a fix for this and haven’t been successful.

    Any ideas?

    Thanks!

    -gregg

  • {"email":"Email address invalid","url":"Website address invalid","required":"Required field missing"}
    >
    Malcare WordPress Security