Nutanix per VM QoS (IOps/Throughput throttling)

In this post I will discuss a new feature to AOS 5.11  called Storage QoS. Storage QoS is essentially rate limiting a specific virtual machine so that it does not exceed a certain IOPs or Throughput threshold.

Now before we dig in and show how Nutanix has implemented QoS and how it can be enabled we must explain that Nutanix has always had storage QoS; albeit hidden and not exposed to the administrators. Within the AOS code there are fair queuing mechanisms to avoid noisy neighbors and adaptive replica placement to ensure drives are properly utilized. Over the years this has been built on by separating queues for reads and writes such that reads are not starved under a heavy write workload. As well as (within the AHV hypervisor) the ability to automatically discover CVM CPU hotspots and dynamic VM migration or iSCSI redirection to ensure consistent performance.

After requests from customers to enable a this QoS dial and allow more control over individual machines this feature was added to the Prism Central management plane. Below I will walk through how we setup and show Storage QoS working on a Nutanix cluster.

1) First I setup a quick IOmeter test to drive up IOps and see what performance I am getting by default. This test is an out of the box IOmeter test using 16 KiB block sizes and 50% random reads. Using this test I have a baseline so when I set Storage QoS I can make sure I set the threshold below this baseline to test the QoS functionality.

2) Next after logging into Prism Central and using the Search feature to find my VM I selected the `Set QoS Attributes` checkbox.

In the following screenshot we can see that I have the option for IOPS or Throughput. For this test I selected IOPs. If you hit ‘More Details’ you will get a really neat chart to guide us on comparing IOps and Throughput to make sure we understand the effects of this throttle.

3) Now that I have set the ‘Throttle IOPS‘ to 1000 I will run the IOmeter test again using the same workload and check the results.

4) In the results we can see that my IOPs never exceeded 1,000 and that my throughput in MiBps match closely what was seen in the chart prior to enabled Storage QoS.

As with all blog posts, I hope this helps bring light to a new and exciting feature available now in the Nutanix Enterprise Cloud!


Share with the world!

Leave a Reply

Your email address will not be published. Required fields are marked *