SQL Server TPC-H-Like Performance on VMware All-Flash Virtual SAN

This is a reference architecture of SQL Server OLAP query on VMWare All-Flash Virtual SAN.

Advantages of running TPC-H query on Virtual SAN:

• All Flash Virtual SAN ensures a desired level of storage performance for bandwidth tantalized OLAP queries and decision support workloads.
• The building block based reference architecture validates All Flash Virtual SAN’s ability to support linear performance on All Flash Virtual SAN

Architecture

This solution architecture is shown in below. We leveraged 4-node cluster to host two SQL Server virtual machines. We enabled HA to provide restart capability of the virtual machine. We enabled deduplication and compression to leverage the All-Flash Virtual SAN space effectively.

Microsoft SQL Server 2014 Virtual Machine Configuration

We configured four SQL Server virtual machines for the performance tests and we created databases per Benchmark Factory for Databases. The scale factor was set to 300 when creating the database. The database and index files consumed approximately 480GB space. We assigned 48 vCPUs and 220GB memory to the VM. We set the maximum server memory to 200GB with Lock Pages in Memory privilege granted to the SQL Server instance startup account. Table 4 lists the SQL Server 2014 VM configuration.

1. Table 1. SQL Server 2014 VM Configuration

Design Best Practices

Minimal usage of the tempdb

To minimize the usage of the tempdb for sorting for the complicated queries of TPC-H-like workload, large percentage of the memory allocation to the SQL Server buffer pool against the database size was designed. In the validation, 60 percent of the working set aka the database size was the desired maximum memory allocation of the SQL instance.

Horizontal table partitioning

We horizontally partitioned ORDERS and LINEITEM table and the partitioning columns are L_SHIPDATE and O_ORDERDATE. The partition granularity is by half year and these two tables are split into 12 partitions. The disk groups for horizontal partitioning reused the disk groups for the data.

The horizontal partitioning for large tables has following advantage for query performance.

• Partition elimination – Partition Elimination is a technique used by query optimizer to not consider partitions that don’t contain data requested by the query. For example, if a query requests data for only the years 1998 and 1999, in that case only two partitions will be considered during query optimization and execution unlike a single large table where query optimizer will consider the whole dataset; the other partitions will be simply ignored.
• Parallel Processing – Query Optimizer uses a technique to process each partition in parallel or even multiple CPU cores can work together to work on a single partition. With this, the query optimizer tries to utilize modern hardware resources efficiently. For example, if a query requests data for only the years 2000 and 2001, in that case only two partitions will be considered during query optimization and suppose if you have 8 cores machine, all 8 cores can work together to produce the result from the two identified partitions.

SQL Server 2014 startup options

The following options are recommended for data warehouse workload of SQL Server 2014:

• -E -- this startup option increases the number of contiguous extents in each file allocated to a database table as it grows and improves the sequential disk access.
• -T117 -- this startup option ensures even growth of all files in a file group when autogrow option is enabled.
• -T834-- this startup option can improve throughput rates for many data warehouse workloads by enabling large page allocation in memory for the SQL Server buffer pool.

Use multiple virtual disks to replace larger stripe width of Virtual SAN

We recommend to fully utilize the physical disks for OLAP read intensive workload. We measured the object distribution on the 32 capacity tier disks.  Using 16 virtual disks with FTT=1 can distribute the object of one virtual machine to 30 disks. That means it is unnecessary to increase the default stripe width of Virtual SAN to the number larger than 1.

Hardware Resources

We used direct-attached SSDs on ESXi server to provide Virtual SAN solution. Each ESXi server has two disk groups each consisting of one cache tier SSD and four capacity tier SSDs. The raw capacity of the Virtual SAN datastore is around 11.88TB.

Each ESXi Server in the Virtual SAN Cluster has the following configuration as shown in Table 1.

1. Table 2. ESXi server Configuration

Software Resources

Table 2 shows the software resources used in this solution.

1. Table 3.    Software Resources

Network Configuration

A VMware vSphere Distributed Switch™ acts as a single virtual switch across all associated hosts in the data cluster. This setup allows virtual machines to maintain a consistent network configuration as they migrate across multiple hosts.

The vSphere Distributed Switch uses two 10GbE adapters for the teaming and failover purposes. A port group defines properties regarding security, traffic shaping, and NIC teaming. We used default port group setting except the uplink failover order as shown in Table 3. It also shows the distributed switch port groups created for different functions and the respective active and standby uplink to balance traffic across the available uplinks.

1. Table 4. Uplink and VLAN settings of the Distributed Switch Port Groups

We used different VLANs to separate he traffic of the vSphere vMotion and Virtual SAN while providing the NIC failover function.

Application Workload

We used the Benchmark Factory for Database to generate TPC-H-like queries.  TPH-H-like queries are for data warehouse workload to support a broad range of queries from a wide-ranging audience such as finance, marketing, operations and research teams. Data warehousing can involve much larger dataset requests and benefit greatly from the increased total throughput potential of sequential disk scans.

Test Result

The solution validates the performance of SQL Server TPC-H-like query against standalone instances in a virtualized VMware environment backed by the Virtual SAN storage platform. 

We measured the performance when run 1 stream on one database only, then run two streams concurrently on two databases separately.

Application performance

The table below shows the test duration for the two virtual machines.  We leverage 1 stream (22 queries) for each database. And all streams could be finished in 26-26 minutes. The CPU utilization was around 21 percent when run one stream of TPC-H-like queries against one virtual machine, and the utilization was around 18 on average when run two one stream of TPC-H-like queries against two virtual machines concurrently.

Virtual SAN performance

We compared the Virtual SAN backend peak IOPS and Bandwidth in GB/sec with 1 virtual machine and the results of two virtual machines.

We compared the Virtual SAN backend peak IOPS and Bandwidth in GB/sec with 1 virtual machine and the results of two virtual machines.

Summary

All-Flash Virtual SAN Cluster provides the capability of a high bandwidth and low latency.

• All-Flash Virtual SAN can achieve up to 2.6GB/sec and more than 50K IOIPS for 2 x 480GB data warehouse databases. With 32 SSD as the capacity tier to support the read intensive workload.
• The two virtual machines can achieve 2 times of the workload of 1 database. The performance incremental was linear and the test duration was keep in 27 minutes to finish.
• The backend Virtual SAN latency was excellent: we achieved less than 7ms read latency and less than 5ms average latency for all queries during test run.