Behind the “the server is powerful but the VM is slow” complaint there is usually not a lack of hardware but wrong sizing. During P2V, the 16 cores of the physical machine are given to the VM as is, memory is doubled “just in case,” and after a while the host starts competing with itself. Below we cover vCPU, memory and NUMA decisions on Hyper-V and VMware vSphere based on how the hypervisor schedules resources; the goal is to increase consolidation without hurting response time.
Why extra vCPUs slow a VM down
The guest operating system sees a vCPU as a physical core. In reality it is a thread that the hypervisor runs in turn on logical processors. When a vCPU wants to run, the hypervisor has to find a free logical processor; if the host is busy, the vCPU waits in the queue. This wait isn’t visible from inside the guest. The application owner only sees slowness.
Extra vCPUs increase this wait in three ways:
- Scheduling pressure: A VM with 8 vCPUs needs more free logical processors at the same time than a VM with 2 vCPUs. vSphere’s “relaxed co-scheduling” mechanism limits how far vCPUs drift apart; when vCPUs that get ahead are stopped, this shows up in esxtop as
%CSTP(co-stop). - Overhead inside the guest: Idle vCPUs also generate timer interrupts, inter-processor interrupts (IPI) and spinlock waits. An application that isn’t multi-threaded doesn’t benefit from extra vCPUs, but it carries this overhead.
- Exceeding the NUMA boundary: If a VM grows too large to fit in one NUMA node, part of its memory is accessed through a remote node; latency rises.
The number of vCPUs is set by the number of threads the application really uses in parallel and the measured peak load. “The physical server had 16 cores” isn’t a measurement. It isn’t even a justification.
vCPU:pCPU ratio and consolidation
The vCPU:pCPU ratio is the total vCPUs assigned to all VMs on a host divided by the number of physical cores. There is no universal “safe” ratio valid for every server and system infrastructure, because what matters is not assigned vCPUs but vCPUs that want to run at the same time. Office applications that sit at low load all day tolerate a high ratio; build servers, SQL Server or RDS session hosts that use the CPU constantly need a low ratio.
Misreadings of the ratio are common at these three points:
- Hyper-Threading: The number of logical processors appears to be twice the physical cores, but the two threads share the execution units of the same core. Base capacity calculations on physical cores and count HT as headroom.
- Hyper-V core scheduler: The core scheduler, the default since Windows Server 2019, gives the two SMT threads of a core only to virtual processors of the same VM. This is a security decision; therefore exposing the SMT topology to the VM (
Set-VMProcessor -HwThreadCountPerCore) and giving an even number of vCPUs makes more efficient use of resources. - Failure scenario: When a host in the cluster goes down, its VMs move to the others. Evaluate the ratio not in the normal state but in the N-1 state. We cover the cluster side separately in the Hyper-V failover cluster planning article.
A ratio that looks comfortable on paper turns into a completely different picture when a node is lost. Do the calculation with N-1.
NUMA and virtual NUMA
On multi-socket servers, each processor has its own attached memory channels; a processor and the memory directly attached to it together form a NUMA node. On some newer processors, more than one NUMA node can be defined even within a single socket (sub-NUMA clustering or “NUMA per socket” settings in the BIOS). Local memory access is faster than accessing the other node’s memory.
The rule is simple: if possible, a VM’s vCPUs and memory should fit in a single NUMA node. On a host with 16 cores and 256 GB memory per node, a 12 vCPU / 128 GB VM can run on a single node; a 24 vCPU VM will necessarily span two nodes. Not noticing that the BIOS NUMA setting changed when renewing hosts is also common; check the number of nodes first thing when moving to new hardware.
Virtual NUMA in Hyper-V
When a VM is larger than a physical node, Hyper-V presents a virtual NUMA topology to the guest; the guest operating system and NUMA-aware applications (such as SQL Server) place memory accordingly. The critical detail: virtual NUMA can’t be used in VMs with Dynamic Memory enabled; the guest sees a single node. That’s why static memory is preferred for large database VMs.
# Show host NUMA nodes and memory per node
Get-VMHostNumaNode
# NUMA spanning setting on the host (default: True)
Get-VMHost | Select-Object NumaSpanningEnabled
# The VM's NUMA limits
Get-VMProcessor -VMName SQL01 |
Select-Object Count, MaximumCountPerNumaNode, MaximumCountPerNumaSocket
If NUMA spanning is turned off, a VM that doesn’t fit in a single node can’t start. Before changing the setting, verify that all VMs fit the node size; otherwise you’ll get a surprise at the next reboot.
vNUMA in vSphere
By default, vSphere presents a vNUMA topology to the guest for VMs with more than 8 vCPUs. If CPU hot-add is enabled on the VM, vNUMA is disabled. Turning on hot-add on large VMs “in case it’s needed” means losing NUMA awareness with a click that looks free. The “cores per socket” value affects licensing and topology as seen by the guest; since its relationship with vNUMA varies by vSphere version, verify the current behavior in Broadcom’s documentation for your version.
To verify from inside the guest, use numactl --hardware or lscpu on Linux and Sysinternals Coreinfo -n on Windows.
Memory: dynamic memory, ballooning and overcommit
The two hypervisors take different paths with memory:
| Topic | Hyper-V | vSphere |
|---|---|---|
| Basic mechanism | Dynamic Memory (Startup, Minimum, Maximum, Buffer) | Ballooning, TPS, compression, swap |
| Assigning more than physical memory | Total assigned can’t exceed physical memory | Total configured can exceed physical memory |
| Reclaim method | Memory hot-add to grow, the balloon driver in the guest to reclaim | Balloon driver (VMware Tools), host swap if needed |
| Lower limit and priority | Minimum RAM and memory weight (priority) | Reservations and shares |
In Hyper-V, Dynamic Memory starts the VM with the Startup value, grows it up to Maximum based on the guest’s demand and the Buffer setting, and reclaims down to Minimum under pressure. If there isn’t enough free memory for Startup during a restart, Smart Paging kicks in and disk is used. This only rescues the boot. It isn’t meant for continuous operation.
In vSphere, memory overcommit is possible; when the host comes under pressure, ballooning comes first, then compression, and finally hypervisor swap. Because hypervisor swap is done without knowing which pages matter to the guest, it hits performance hard. If you see “swapped” or “compressed” memory on the monitoring dashboard, the discussion is over: host memory capacity is insufficient.
Constraints on the application side directly affect sizing. Exchange Server isn’t supported with dynamic memory. SQL Server can run with dynamic memory; but a SQL instance without a max server memory setting tends to hold on to the memory it is given, so ballooning has no effect. For large SQL and application servers, static memory with a realistic max server memory value gives more predictable results.
# Hyper-V: dynamic memory example for a file server
Set-VMMemory -VMName FS01 -DynamicMemoryEnabled $true `
-StartupBytes 4GB -MinimumBytes 2GB -MaximumBytes 8GB -Buffer 20
# Static memory for a SQL server (with the VM powered off)
Set-VMMemory -VMName SQL01 -DynamicMemoryEnabled $false -StartupBytes 64GB
Which counter to look at
On a Hyper-V host, the CPU usage shown by Task Manager is only the view of the root partition, i.e. the host operating system. It doesn’t show the host’s real processor load. The diagnosis “the host is at 20 percent, the problem is in the VM” often comes from here, and it’s wrong. The right source is the hypervisor counters:
| Platform | Counter / metric | What it tells you |
|---|---|---|
| Hyper-V | Hyper-V Hypervisor Logical Processor(_Total)\% Total Run Time |
Real utilization of the physical processors |
| Hyper-V | Hyper-V Hypervisor Virtual Processor(*)\CPU Wait Time Per Dispatch |
Time a vCPU waits to run (in 100 ns units) |
| Hyper-V | Hyper-V Dynamic Memory VM(*)\Average Pressure |
Ratio of the VM’s memory demand to assigned memory |
| vSphere | CPU Ready (%RDY), Co-stop (%CSTP) |
Time a vCPU was ready but kept waiting |
| vSphere | Balloon, Swapped, Compressed memory | Host memory pressure |
In vCenter performance charts, CPU Ready is shown as a sum in milliseconds. Example calculation to convert to a percentage: in the real-time chart the sampling interval is 20 seconds; a ready value of 1,000 ms means 1,000 / 20,000 = 0.05, i.e. 5 percent of that interval. This value is the total for all of the VM’s vCPUs; divide by the number of vCPUs to evaluate it. Look at the trend during business hours, not a single instantaneous value.
# Collect 12 samples at 5-second intervals on a Hyper-V host
Get-Counter -Counter `
'\Hyper-V Hypervisor Logical Processor(_Total)\% Total Run Time',
'\Hyper-V Hypervisor Virtual Processor(*)\CPU Wait Time Per Dispatch' `
-SampleInterval 5 -MaxSamples 12
For host-level thresholds and alert rules, see the server monitoring metrics article.
Starting values by workload type
The values below are reasonable assumptions for the first deployment when there are no measurements; they aren’t a standard. Every VM is re-evaluated after the first two to four weeks of measurement.
| Workload | vCPU | Memory | Note |
|---|---|---|---|
| Domain controller / DNS | 2 | 4–8 GB | The AD database should fit in memory |
| File server | 2 | 4–8 GB | Dynamic memory suitable |
| Application / web server | 2–4 | 8–16 GB | Follow the vendor’s requirements |
| SQL Server (medium) | 4–8 | 32–64 GB | Static memory, single NUMA node |
| RDS session host | 8–16 | 32–64 GB | Scales with the number of users |
For products licensed per core, such as SQL Server, the number of vCPUs directly determines licensing cost. Extra vCPUs here are both a performance and a budget mistake. We look at the licensing side separately in the Windows Server licensing for virtualization article.
Resizing: the order of checks
- Take an inventory: List assigned vCPUs, memory, dynamic memory status and host NUMA placement for each VM.
- Measure for at least two weeks: Collect in-guest CPU usage, processor queue length and hypervisor wait counters with data covering peak periods such as month-end and the backup window.
- Separate the candidates: VMs with many vCPUs whose peak CPU usage stays consistently low are candidates for shrinking; those with both high wait time and high in-guest usage are candidates for growing. If usage in the guest is low but the wait is high, the problem isn’t in the VM but in the host.
- Change gradually: Instead of halving vCPUs at once, shrink by one step. On Hyper-V, vCPU changes are made with the VM powered off; plan the maintenance window accordingly.
- Verify: Compare the same counters and application response times before and after the change, and add the result to the change record.
# Quick inventory of assigned resources
Get-VM | Select-Object Name, ProcessorCount,
@{n='StartupGB';e={$_.MemoryStartup/1GB}},
@{n='AssignedGB';e={[math]::Round($_.MemoryAssigned/1GB,1)}},
DynamicMemoryEnabled | Sort-Object ProcessorCount -Descending
# Reduce vCPUs (with the VM powered off)
Stop-VM -Name APP02
Set-VMProcessor -VMName APP02 -Count 4
Start-VM -Name APP02
Common mistakes
- Carrying the physical server’s core and memory values over to the VM as is after P2V.
- Reading host CPU usage from the root partition’s Task Manager.
- Turning on dynamic memory on a large SQL VM and unknowingly disabling virtual NUMA.
- Planning capacity for the normal state and not accounting for the loss of a cluster node.
- Giving a VM a single vCPU: during updates or an antivirus scan the whole system gets stuck on one core; at least 2 vCPUs is healthier for most server workloads.
For capacity and performance analysis of existing hosts, see our infrastructure consulting and system optimization page; for platform design, see our virtualization and data management service. Microsoft’s official recommendations are in the Hyper-V performance tuning guide.
Frequently asked questions
Can the number of vCPUs given to a VM exceed the number of physical cores?
A single VM can’t be given more vCPUs than the number of logical processors on the host. Thanks to Hyper-Threading, a value above the number of physical cores can be given, but it isn’t recommended; when all of the VM’s vCPUs want to run at the same time, it competes with its own vCPUs. Total vCPUs across the host exceeding physical cores is normal, however; that is what consolidation is.
If the CPU Ready value is high, is adding vCPUs to the VM the solution?
Usually not. CPU Ready shows that a vCPU was ready to run but couldn’t find a physical processor. Adding vCPUs increases the need to wait. The right step is to shrink VMs with too many vCPUs, spread the load to another host or increase host capacity.
Which servers is Dynamic Memory not suitable for?
Exchange Server isn’t supported with dynamic memory. For large SQL Server VMs, static memory is preferred because dynamic memory disables virtual NUMA and SQL tends to hold on to memory. Variable loads such as file servers, web servers and management servers are good candidates for dynamic memory.
How do I know whether a VM fits in a NUMA node?
On Hyper-V, Get-VMHostNumaNode shows cores and memory per node, and Get-VMProcessor shows the VM’s limits. If the VM’s vCPU count is smaller than the physical cores in a node and its memory is smaller than the node’s memory, it fits in a single node. The number of nodes seen from inside the guest with Coreinfo -n or numactl –hardware is the second check.