SPECaccel® 2023 Benchmark Suite — A Teaching Tool for High Performance Computing

By Mathew Colgrove, SPEC HPG Release Manager

I'm very pleased to announce the availability of the SPECaccel 2023 benchmark suite. A major update to the SPEC ACCEL v1.4 benchmark suite, the SPECaccel 2023 benchmark suite tests the performance of key system components, including Graphics Processing Units (GPUs) and Accelerated Processing Units (APUs), using computationally intensive parallel applications using directives, OpenACC and OpenMP target offloading APIs. The new suite uses 12 of the original v1.4 benchmarks with larger, up to 16GB, workloads to better measure modern accelerators.

The development of the SPECaccel 2023 benchmark suite is a collaboration by representatives from industry vendors, high performance computing centers and academic institutions to provide a baseline to compare accelerator performance. While vendor supplied languages may offer optimal performance for a particular accelerator, the use of directives showcases performance portability across many accelerators.

The suite includes multiple methods to apply the directives with the choice of method made by the user based on which compiler they are using and the target architecture. Given the benchmarks are proxy applications representing several scientific domains, the suite can help users understand how the different methods could then be applied to their own applications. For "peak" runs, users are allowed to modify the directives for further tuning.

The benchmarks use OpenMP metadirectives to choose between three methods:

• The "loop" directive with collapsing of nested loops.
• The "distribute/parallel" directives with full collapsing of nested loops.
• The "distribute/parallel" directives with collapsing of outer loops and use of "simd" on inner loops

Given compiler support of metadirectives is still emerging, the suite provides alternate sources for each method without metadirectives. Several benchmarks also include alternate sources using only the OpenACC "parallel" construct for compilers with limited support of the "kernels" construct.

Another important area where the benchmark suite shines is academia. Instructors may use the SPECaccel 2023 benchmark suite for educational purposes, including the study of directive based parallel programing models, OpenACC and OpenMP. An instructor could, for example, design an entire course to explore the principles and practices of high-performance computing, using the benchmark suite as a foundation for evaluating and optimizing the performance of systems. Students could use the benchmark suite to test different GPU-accelerated systems, execute tests and interpret results. They could also undertake optimization projects where they use the benchmark suite to measure the impact of the changes they make. They could compare the performance of different GPU architectures as well as explore the relationship between different GPUs and different types of workloads. Finally, students could also use the benchmark suite to see how their code design decisions related to parallel programming principles impacts performance.

Qualified academic and research users can apply for a no-cost license.

For commercial use, the benchmark suite is available for immediate download from SPEC for $2,000.

I'd like to also encourage you and your organization to consider joining SPEC's High Performance Group (HPG) and help with our development efforts. We primarily focus on benchmarking high performance computing applications using standard parallel models. While challenging, it's also an exciting and rewarding experience. For educational institutions, it's a great opportunity for your students to collaborate directly with industry professionals and researchers from national labs. Our SPEChpc Next project is just starting, so it's a great time to join!

For more information about the SPECaccel 2023 benchmark suite, please visit the SPEC website.

[Back to SPEC blog]