ESXi Architecture

This is my summary of the following VMware article:

www.vmware.com/files/pdf/vmware_esxi_architecture_wp.pdf




ESXi Components

1. VMkernel

POSIX like operating system. It provides functionality such as process creation & control, signals, file system and process threads. It provides resource scheduling, I/O stacks and device drivers to enable the running of multiple Virtual Machines.

2. File System

The VMkernel provides a simple in memory file system to hold config file, log files and staged patches. The file system layout is similar to the ESX Service Console.

The file System is independent of the VMFS file system, and if all VMFS file systems for the host are on shared storage, then the host can be a diskless machine. However if the host will be a member of an HA cluster, refer to this KB article as you will need to configure the location of your swap file:

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1004177

Remote access to the file systems is manage by HTTP get and put requests with authentication against local groups and users privileges.

The file system is not persistent, and thus log files will be lost on reboot, so it is definitely worth configuring a syslog server. However, for ESXi embedded, configuration information is written to a readable and writable memory location. This memory is persistent, and is read from on boot.

3. Users and Groups

They can be defined locally for use with the Vi Client, VIM API, or remote command line.

4. User Worlds

"User World" is a process running in the VMkernel environment. The user world environment is very limited compared to normal POSIX-like environments. For Example:

• The set of available signals is limited.
• The system API is a subset of POSIX.
• The /proc file system is very limited.
• A single swap file is available for all user world processes. If a local disk exists, the swap file is created automatically in a small VFAT partition. Otherwise, the user is free to set up a swap file on one of the attached VMFS datastores, by changing the advanced paramenter ScratchConfig.ConfiguredScratchLocation, see KB link above for more details. For each host, the file is about 1GB in size, and each ESXi host will require a unique directory name for its swap file location. Probably the best approach is to have a single LUN for all swap files (say 10GB for 8 hosts).

The user world is a very limited set of functionality aimed at only running the processes required, and no more.

5. Direct Console User Interface

Runs in a user world. It is used for configuration and management interface through the system console. This is used for initial basic configuration of the host. It uses the system user dcui in the VMkernel so it can identify itself when communication with other processes. The DCUI can:

• Set administrator password
• Configure netowrking, or set it to DHCP
• Perform some network tests
• View logs
• Restart agents
• Restore defaults

it is possible to give individual users access to the DCUI by adding them to the localadmin group, thus removing the need to hand out the root password.

6. Other User World Processes

• hostd - provides interface to VMkernel. Tracks users & groups privileges and provides authentication.
• vpxa - used for connection to vCenter. Runs as special system user vpxuser and interfaces between hostd and vCenter.
• HA agent - runs in its own user world
• syslog daemon - used to forward logs to syslog server
• iSCSI discovery - is performed in its own user world. After discovery, the VMkernel handles all traffic. The iSCSI network interface is the same as the main VMkernel network interface.
• NTP - has process to manage NTP time synchronisation
• SNMP - has process to manage SNMP monitoring & alerts

7. Startup

Any upgrade to ESXi can be easily performed because the ESXi image is only 32MB, and thus will just be replaced for upgrades or patching. There are 2 banks for ESXi packages, and either can be used to boot the ESXi host. So if an upgrade is performed and there is a problem, the previous package can be used from the alternate bank. This can be either automatic, or the administrator can choose which bank to boot from.

8. CIM

The Common Information Model (CIM) is an open standard that defines how computing resources can be represented and managed. It enables a framework for agentless, standards-based monitoring of hardware resources for ESXi. This framework consists of a CIM object manager, often called a CIM broker, and a set of CIM providers.

The CIM providers provide a way gain management access to device drivers and hardware. Both hardware manufacturers and VMware have written specific CIM providers. These providers are packaged with ESXi and can be installed at runtime. The CIM broker take all the information from the CIM providers and presents it via standard APIs.

9. VI API

The VI API provides a common interface for vCenter, ESX and ESXi enabling bespoke applications and functionality to be developed, but obvioulsy certain functions will only work with certain targets.