SPEC Cloud IaaS 2016 Benchmark - FAQ and Glossary¶
Contents:
- Benchmark Overview
- 1) What is SPEC Cloud IaaS 2016 benchmark?
- 2) Who is the intended audience for the benchmark?
- 3) What are the primary objectives and goals of the SPEC Cloud IaaS 2016 Benchmark?
- 4) Can SPEC Cloud IaaS 2016 Benchmark be used for PaaS or SaaS environments?
- 5) Where can I find the glossary of terms used in the benchmark?
- 6) What are the commonly used terms in this FAQ?
- 7) What is the system under test in SPEC Cloud IaaS 2016 benchmark?
- 8) Why should one use SPEC Cloud IaaS 2016 Benchmark as opposed to other benchmarks used to evaluate cloud performance?
- 9) What are the limitations of SPEC Cloud IaaS 2016 Benchmark?
- 10) Can I use SPEC Cloud IaaS 2016 Benchmark to determine the size of the cloud I need?
- 11) Does SPEC Cloud IaaS 2016 Benchmark have a power measurement component associated with the benchmark?
- 12) Can I compare SPEC Cloud IaaS 2016 Benchmark results for private (white box) and public (black box) clouds?
- 13) Which cloud platforms are supported by SPEC Cloud IaaS 2016 Benchmark?
- 14) What skills do I need to run SPEC Cloud IaaS 2016 Benchmark?
- 15) How can I use the benchmark to research performance related to a specific component of the system under test such as the memory, storage, hypervisor, application server, virtual machine, or containers?
- 16) A public cloud may comprise thousands of machines. A tester paying for usage of these resources within the cloud may incur a huge bill while running the benchmark.
- Relationship to Other SPEC Benchmarks
- Metrics
- Workloads Used
- Benchmark Kit
- 25) How can I obtain the SPEC Cloud IaaS 2016 benchmark kit?
- 26) What is included with the SPEC Cloud IaaS 2016 Benchmark kit?
- 27) Where can I find a user guide to run the benchmark?
- 28) What if I followed the user guide and have questions running SPEC Cloud IaaS 2016 Benchmark?
- 29) Where are the run and reporting rules for SPEC Cloud IaaS 2016 Benchmark?
- Testing, Compliant Run, Results Submission, Review and Result Announcement
- 30) How can I submit SPEC Cloud IaaS 2016 Benchmark results?
- 31) When and where are SPEC Cloud IaaS 2016 Benchmark results available?
- 32) Can I report results for newer versions of Apache Hadoop and Cassandra?
- 33) Are the results independently audited?
- 34) Can I announce my results before they are reviewed by the SPEC OsgCloud subcommittee?
- 35) Are results sensitive to components outside of the SUT e.g. client driver machines?
- 36) What is a “compliant” result of SPEC Cloud IaaS 2016 Benchmark?
- 37) Can I run other workload levels?
- 38) How long does it take to run the benchmark?
- 39) How long does it take to setup the benchmark?
- 40) Can I declare my SPEC submission as white box without having full admin access and visibility into the hypervisors and cloud management software?
- 41) Can I declare my SPEC submission as black box without it being generally available to other users?
- 42) Can I declare my SPEC submission as both black box and private cloud?
- 43) If I declare my SPEC submission as black box am I allowed to make hardware customizations or service prioritization?
- 44) If I make hardware customizations or service prioritization in my cloud can I declare my SPEC submission as white box?
- Glossary of terms (sorted alphabetically)
