One of the things that working in performance enables is the ability to play with a wide variety of environments, ranging from vSphere to public cloud providers. Unsurprisingly, making comparisons across these dissimilar environments has historically been very difficult. Some time back I started crafting a benchmark harness that would allow these kinds of comparisons to be more easily made. After a bit of tinkering, a benchmark named StratusMark was introduced within VMware. Since then we’ve used it for several internal case studies involving environment comparisons. After talking with a few colleagues recently about how the benchmark might be more broadly used, I thought now might be a great time to provide some context around StratusMark, our usage of it, and some of the data we hope to collect in the future.
StratusMark is a Java-based application. It is capable of provisioning a wide variety of workloads across a wide variety of environments.
StratusMark Environment Options:
- vSphere
- vCloud Director
- vRealize Automation
- Amazon AWS
- Microsoft Azure
- Google Compute
- IBM Softlayer
- OVH (formerly vCloud Air)
It also has several modes of operation, allowing the deployment of various types of workloads.
StratusMark Operating Modes:
- Lifecycle Measurements
- Microbenchmarking
- Macrobenchmarking
For lifecycle measurements, it captures the time between a resource request, through a steady state, and then to termination of a resource. Microbenchmarking generally utilizes existing templates (or images) already available to an environment. This mode allows the utilization of smaller benchmarks to be transferred as part of an instance’s payload. This payload is user-definable; it can be as simple as a “ls” command to verify an instance is truly ready for work to something as complex as SPECjbb. Macrobenchmarking leverages user-modified templates to introduce instance-to-instance dependencies. For example, when running DVD Store 3 there are instance dependencies between the web tiers and the database.
As seen from the above environment and operating lists, StratusMark can deploy and compare a diverse set of workloads in interesting ways. Stay tuned for additional posts highlighting some of these comparisons.