When a “VMs are slow” complaint comes in on a Hyper-V cluster, the first place to look is often not the CPU but the storage path. Which protocol and which network hold the VM disks directly determines performance, failure behavior and day-to-day operational load. This article compares iSCSI, SMB3 and Fibre Channel for Hyper-V clusters that will use external storage, and gives MPIO, SMB Multichannel and Cluster Shared Volume (CSV) configuration with PowerShell examples. Read it together with the Hyper-V failover cluster planning article for the cluster’s general design checks.
Storage access models in a Hyper-V cluster
A Hyper-V cluster accesses VM files in two basic ways:
- Block storage (iSCSI, Fibre Channel, SAS): The array presents LUNs to all nodes; the LUNs are formatted with NTFS or ReFS and added to the cluster as CSVs. VMs live under
C:\ClusterStorage\VolumeN. - File storage (SMB 3.x): VM files live on a share on a file server (usually a Scale-Out File Server or an SMB3-capable NAS); Hyper-V hosts access them by UNC path.
The third model, Storage Spaces Direct (S2D), uses no external storage and pools the hosts’ local disks. The architectural decision between HCI and classic SAN is in the HCI and SAN decision guide; here we cover only the protocol for accessing external storage.
iSCSI: network design and MPIO
iSCSI carries block storage over standard Ethernet (TCP 3260). Designed correctly, it gives enough performance and redundancy for enterprise workloads. Designed wrongly, it becomes the source of latency affecting all VMs in the cluster at once, and the problem is often looked for in the wrong place, in the application or the hypervisor.
Basic design rules:
- At least two dedicated physical ports for iSCSI on each host; these ports aren’t shared with VM traffic or the management network.
- The two ports are not teamed; redundancy and load distribution come from MPIO. Each port is on a separate subnet (e.g. 10.20.1.0/24 and 10.20.2.0/24) and, if possible, a separate switch.
- No default gateway is defined on the iSCSI interfaces, and DNS registration is turned off; the Client for Microsoft Networks and File and Printer Sharing bindings can be removed.
- If jumbo frames are used, the same MTU is set on the host NIC, the switch ports and the storage array. A single missing point causes fragmentation or connectivity problems.
A common iSCSI error is forgetting the MTU at one point on the path. Small packets get through, large ones get stuck. To check, send a large unfragmented packet from the host to the array with ping -f -l 8972; if there’s no reply, a link in the chain is missing.
MPIO setup and the iSCSI connection are done with PowerShell like this (addresses are examples):
Install-WindowsFeature -Name Multipath-IO # requires a reboot
Enable-MSDSMAutomaticClaim -BusType iSCSI
Set-MSDSMGlobalDefaultLoadBalancePolicy -Policy RR
Set-Service msiscsi -StartupType Automatic; Start-Service msiscsi
New-IscsiTargetPortal -TargetPortalAddress 10.20.1.50 -InitiatorPortalAddress 10.20.1.11
New-IscsiTargetPortal -TargetPortalAddress 10.20.2.50 -InitiatorPortalAddress 10.20.2.11
$t = (Get-IscsiTarget).NodeAddress
Connect-IscsiTarget -NodeAddress $t -TargetPortalAddress 10.20.1.50 `
-InitiatorPortalAddress 10.20.1.11 -IsMultipathEnabled $true -IsPersistent $true
Connect-IscsiTarget -NodeAddress $t -TargetPortalAddress 10.20.2.50 `
-InitiatorPortalAddress 10.20.2.11 -IsMultipathEnabled $true -IsPersistent $true
For the load balancing policy (Round Robin, Least Queue Depth, etc.), follow the storage vendor’s recommendation; some arrays require their own DSM or a specific policy. Verify the number of paths with mpclaim -s -d: the expected number of paths should appear for each LUN. If only one path appears, there is no MPIO.
SMB3: Multichannel, RDMA and Scale-Out File Server
SMB 3.0, introduced with Windows Server 2012, supports keeping Hyper-V VM files on a file share. Two features make this scenario strong:
- SMB Multichannel: If there are multiple NICs, SMB automatically opens multiple connections; it aggregates bandwidth and keeps the session going if a NIC fails. It needs no extra configuration.
- SMB Direct (RDMA): On iWARP, RoCE or InfiniBand-capable NICs, data is transferred largely bypassing the CPU. If RoCE is used, DCB/PFC configuration is needed on the switch side; iWARP doesn’t have this requirement. Half-done PFC is worse than not enabling RDMA at all.
For high availability on the file side, the Scale-Out File Server (SOFS) role is used. SOFS serves the share on the CSV from all file server nodes at once; thanks to “continuously available” shares, Hyper-V hosts continue I/O without interruption if a node fails. The essence of the setup:
Add-ClusterScaleOutFileServerRole -Name SOFS01
New-Item -ItemType Directory -Path C:\ClusterStorage\Volume1\Shares\VMs
New-SmbShare -Name VMs -Path C:\ClusterStorage\Volume1\Shares\VMs `
-FullAccess 'CORP\HV01$','CORP\HV02$','CORP\HV-CLU$','CORP\HyperV-Admins' `
-ContinuouslyAvailable $true
Set-SmbPathAcl -ShareName VMs # applies share permissions to the folder NTFS ACL
Both share and NTFS permissions must be granted to the Hyper-V hosts’ computer accounts and the cluster name. If VMs are created from a remote management machine, Kerberos constrained delegation must be configured; otherwise you get an “access denied” error, and hours are lost because the permissions look correct. Verification:
Get-SmbMultichannelConnection
Get-SmbClientNetworkInterface
Get-NetAdapterRdma | Where-Object Enabled
Third-party SMB3-capable NAS devices can also be used. Verify that they support the SMB 3 features Hyper-V needs (especially continuous availability) and that the vendor has certified them for Hyper-V.
When Fibre Channel makes sense
Fibre Channel requires a separate physical network (HBAs, FC switches, zoning). Setup and operation need additional expertise; in return it gives a lossless structure with predictable latency that separates storage traffic entirely from Ethernet. FC usually makes sense when:
- The organization already has a working FC fabric and a team that can manage it
- Very latency-sensitive, high-I/O databases use the same array
- Guest VMs need to see LUNs directly: Hyper-V’s Virtual Fibre Channel feature gives a VM a virtual HBA through NPIV (the HBA and switch must support NPIV)
In small and mid-sized clusters built from scratch, 25 GbE iSCSI or SMB3 usually delivers the benefit FC brings, with lower operational overhead for server and system infrastructure. Most FC failures come from zoning and HBA firmware incompatibility; if no one will track the firmware matrix, don’t deploy FC.
Protocol comparison table
| Feature | iSCSI | SMB3 | Fibre Channel |
|---|---|---|---|
| Transport | Ethernet, TCP | Ethernet, TCP or RDMA | Separate FC network |
| Path redundancy | MPIO | SMB Multichannel | MPIO |
| How it appears in the cluster | CSV (blok) | UNC share | CSV (blok) |
| Additional hardware | Dedicated NICs | RDMA optional | HBAs, FC switches |
| Operational load | Intermediate | Low to medium | High |
| Typical error | Teaming, MTU mismatch | Permissions, delegation | Zoning, firmware |
Cluster Shared Volume behavior
A CSV lets all nodes write to the same LUN at the same time. Each CSV has a coordinator (owner) node; metadata operations (such as creating or extending files) go through the coordinator, while normal reads and writes go directly from each node to the array (Direct I/O).
If a node loses direct access to the LUN, I/O is redirected over the cluster network to the coordinator (redirected I/O). VMs keep running, performance drops noticeably. Nobody gets an alarm. Monitor the state like this:
Get-ClusterSharedVolume | Select-Object Name, OwnerNode, State
Get-ClusterSharedVolumeState | Select-Object Name, Node, StateInfo, FileSystemRedirectedIOReason, BlockRedirectedIOReason
The StateInfo value should be Direct on all nodes. One detail: CSVs formatted with ReFS on a SAN run in file system redirected mode; that’s why NTFS is recommended for CSVs on external block storage. ReFS running in direct mode on a CSV is specific to the S2D scenario. Formatting a SAN LUN with ReFS because “the new file system is better” is one of the classic causes of performance complaints in the field.
Plan the number of CSVs based on the number of nodes: creating several CSVs and distributing their ownership across nodes keeps the metadata load from piling up on one node.
Verification and performance testing
Do two kinds of verification before going into production:
- Cluster validation:
Test-Clusteris run including the storage tests. Since storage tests can take disks offline, on a production cluster target only newly added disks that aren’t in use yet. - Load test: Run a realistic profile on the CSV with Microsoft’s DiskSpd tool and look at the latency distribution, not the average:
diskspd.exe -c50G -d120 -b8K -o16 -t8 -r -w30 -Sh -L C:\ClusterStorage\Volume1\test.dat
Repeat the test with two paths and with one path (with one iSCSI port shut down); confirm that I/O isn’t interrupted when a path fails and that latency stays acceptable. Keep these results; when a “it used to be fast” complaint comes later, they will be the only concrete reference you have. We can work together on choosing the storage array and server hardware within our hardware consulting service, and on cluster design and commissioning within our virtualization and data management service. For the virtual switch and traffic separation side, see the Hyper-V SET and virtual network design article.
Frequently asked questions
Can I use NIC teaming for iSCSI?
Microsoft recommends MPIO instead of NIC teaming for iSCSI traffic. MPIO manages storage paths at the iSCSI session level; it both distributes load and provides fast failover when a path fails.
Is SMB3 storage slower than iSCSI?
Not necessarily. With SMB Multichannel and especially RDMA-capable NICs, SMB3 can deliver performance comparable to block protocols. Network design, the file server and disk capacity determine the result; measure it in your own environment with DiskSpd.
Why does a CSV switch to redirected I/O mode?
A CSV switches to redirected mode when a node’s direct path to the LUN is lost, during some backup operations, or when ReFS is used on a SAN. The first place to look in diagnosis is the reason fields in the Get-ClusterSharedVolumeState output.
Is Fibre Channel necessary in a small cluster?
Usually not. Without existing FC infrastructure and staff, iSCSI or SMB3 on a dedicated network gives enough performance with lower operational overhead. FC makes sense in environments with an existing fabric or very latency-sensitive workloads.