SPEChpc™ 2021 System Requirements

$Id$ Latest: www.spec.org/hpg/hpc2021/Docs/

Contents

Overview

I. Introduction

II. Hardware

A. CPUs

B. Memory

C. Disk

III. Software

A. Operating System

B. Compilers or binaries

C. MPI and Ranks

IV. Install media

V. Don't be root. Turn privileges off. (Usually.)

VI. Problems?

System Requirements Overview
CPU ARM, Power ISA, or x86_64. Limitations apply. See: Supported Toolsets.
Physical Memory Tiny 60 GB May vary: memory detail.
Small 480 GB
Medium 4 TB
Large 14.5 TB
Disk Space Recommended 50 GB. on a local or shared network file system. May vary: see disk detail.
Operating System Linux. Limitations apply. See: Supported Toolsets.
Compilers

C11, C++14, Fortran-2008.

or

Pre-compiled benchmarks.

All 3 are required for reportable. See: Compilers and Precompiled binaries.
MPI

MPI-3 compliant MPI installation configured for use with your compilers.

See: MPI.
 

I. Introduction: Suites and Metrics

SPEChpc 2021 includes source code and data sets for 9 benchmarks in 4 suites which each suite being a different workload size targeting different sized systems.

Short
Tag
Suite Contents Metrics How many ranks?
What do Higher Scores Mean?
Tiny SPEChpc 2021 Tiny Workload 9 benchmarks SPEChpc 2021_tny_base
SPEChpc 2021_tny_peak
The Tiny workloads use up to 60GB of memory and are intended for use on a single node using between 1 and 256 ranks. More nodes and ranks may be used however higher rank counts may see lower scaling as MPI communication becomes more dominant.
Higher scores indicate that less time is needed.
Small SPEChpc 2021 Small Workload 9 benchmarks SPEChpc 2021_sml_base
SPEChpc 2021_sml_peak
The Small workloads use up to 480GB of memory and are intended for use on one or more nodes using between 64 and 1024 ranks. More ranks may be used however higher rank counts may see lower scaling as MPI communication becomes more dominant.
Higher scores indicate that less time is needed.
Medium SPEChpc 2021 Medium Workload 6 benchmarks SPEChpc 2021_med_base
SPEChpc 2021_med_peak
The Medium workloads use up to 4TB of memory and are intended for use on a mid-size cluster using between 256 and 4096 ranks. More ranks may be used however higher rank counts may see lower scaling as MPI communication becomes more dominant.
Higher scores indicate that less time is needed.
Large SPEChpc 2021 Large Workload 6 benchmarks SPEChpc 2021_lrg_base
SPEChpc 2021_lrg_peak
The Large workloads use up to 14.5TB of memory and are intended for use on a larger clusters using between 2048 and 32,768 ranks. More ranks may be used however higher rank counts may see lower scaling as MPI communication becomes more dominant.
Higher scores indicate that less time is needed.
The "Short Tag" is the canonical abbreviation for use with runhpc, where context is defined by the tools. In a published document, context may not be clear.
To avoid ambiguity in published documents, the Suite Name or the Metrics should be spelled as shown above.

You may choose which suite(s) you would like to run (there is no requirement to run all of them), and your choice affects hardware requirements and memory

Having chosen a suite, if you will use your results in public, then you must run all the benchmarks in the suite (exceptions) and produce at least the base metric. The peak metric is optional. If producing both base and peak, you will need more disk space.

II. Hardware requirements

II.A. CPUs

II.A.1 CPUs supported

SPEC supplies toolsets for ARM, Power ISA, or x86_64.
Limitations apply:

Although SPEChpc suites are intended to be useful with a wide range of chip architectures, in some cases it is possible that you may find that your chip is not compatible with the available toolsets (for example, if your chip is too old to run one of the supported OS+chip combinations). See the section on Supported Tools.

II.B. Memory

The nominal main memory requirements for each workload are:

(In this section, 1 GB = 2^30 bytes and 1 TB = 2^40 bytes)

Warnings:

II.C. Disk space

The disk space recommendations below are only estimates. Your environment may differ.

The SPEC tools (and optionally the output_root directory) must be installed on a shared network file system when using more than one node.

Estimated disk usage
50 GB
1 GB Install SPEChpc 2021
3 GB Build base for Small suites using 5 different builds and runs.
11 GB Build base+peak for Medium suites each using 10 different builds runs.
Each build and run (base or peak, different parallel models, different compiler flags, etc.) takes approximately 0.4 GB.
Your usage may differ, due to hardware, operating system, disk type, file system type, compiler, and, especially, compiler tuning.
Once you know the space consumption pattern for your hw/sw, you might adjust the above calculations to be more accurate for your needs.

III. Software requirements

III.A. Operating System

The SPEChpc 2021 toolset relies on several open source components, including GNU Make, Perl, and others. SPEC supplies pre-built versions of these for particular combinations of hardware and operating system, as shown in the table below.

Supported Toolsets for SPEChpc 2021
Toolset Intended Use
linux-aarch64 64-bit AArch64 systems running Linux.
linux-armv7l Linux systems with ARM Cortex-A7-compatible CPUs.
linux-ppc64le 64-bit little-endian PowerPC Linux systems
linux-x86_64 x86_64 Linux systems, linked using libnsl.so.1
linux-x86_64-rhel8 x86_64 Linux systems, linked using libnsl.so.2

Limitations apply: Although SPEC has tested the above, it is possible that you may enounter an OS+chip combination that is not compatible with its intended toolset. In such cases:

libnsl.so:In order to be compatible with older Linux OS, the tools are linked against libnsl.so.1 which has since been depricated. However, most newer OSs install it for compatibility or simply link libnsl.so.1 to libnsl.so.2. If the 'linux-x86_64' tools package is used, but you get runtime errors from the tools, such as 'specperl', check to see is libnsl.so.1 is installed on your systes.

Note that RHEL 8 does not provide this compatibility by default. You will need to install libnsl.so.1, or use the linux-x86_64-rhel8 toolset via "install.sh -u linux-x86_64-rhel8".

What about other systems? For systems that are not listed in the table of supported toolsets:

III.B. Compilers or binaries

SPEC supplies the benchmarks in source code form. Therefore, you will need either:

  1. A set of compilers. All three are needed:
    1. C11
    2. C++14
    3. Fortran 2008

    --or--

  2. A pre-compiled set of benchmark binaries (executables), given to you by another user of the same revision of SPEChpc 2021.
    Binaries generated by a different revision of SPEChpc 2021 will not work.
    You might also need a set of run-time libraries for those executables.
    Note that even if someone else compiles the benchmarks, the tester is responsible for compliance with the rules: do not accept binaries from untrusted sources.

Config file

III.C. MPI and Ranks

III.C.1 MPI-3 Installation

The SPEChpc benchmarks use the MPI-3 standard. Therefore, you will need a installation of MPI that supports MPI-3 and is configured for the compilers you indend to use.

III.C.2 Nodes and Ranks

You can use SPEChpc 2021 to measure the compute performance of a system (node) or multiple interconnected systems (cluster) with physical or virtual CPUs/processors. Optionally, attached accelerator devices may be used with OpenACC or OpenMP. You can choose to measure all of the processors on a system, one or more nodes cluster, or a subset.

Typical: All nodes using all physical CPUs

Usually SPEChpc has been used to measure entire clusters, with all of the physical CPU chips.

Alternatives: Accelerators, Single Node, partition, zone,...

There is no prohibition against using SPEChpc to measure a subset of the nodes on a cluster.

If you use a subset, it must have enough memory and disk. For public results, follow the usual rules. (Examples: use only methods that are documented, supported, and generally available to customers; fully disclose what you do, with sufficient detail so that the result can be reproduced; and if you enhance performance by doing something outside the subset

How many Ranks?

You can use SPEChpc 2021 to measure performance with a large numbers of nodes and ranks. However, since SPEChpc is strong scaled, you may see performance degradation as the MPI rank count is increased. As the number of nodes is increased, it is suggested to move to using the Medium or Large workloads.

SPEChpc 2021 approximate rank scaling by workload when using only MPI parallelism.
Tiny Small Medium Large
1-256 64-1024 256-4096 2048-32768

Note that this is only a suggested range. You may use a smaller number of ranks, provided the nodes have enough memory. Larger numbers of ranks can be used but you may see lower scaling.

With accelerators, scaling may decrease at smaller ranks counts.

With multiple CPU threads enabled, scaling may decrease at smaller ranks counts as well depending upon the number of threads used per rank.

Due to domain decomposition, some benchmarks may achieve better performance with a base-2 number of ranks. However, most benchmarks should be able to run at any arbitary number of ranks. The exceptions being MiniSweep which may give incorrect answers when using odd rank counts (except 1 and 3 ranks) and LBM which may give incorrect answers when using ranks counts 2 to 4 times the top scaling given above.

IV. Install Media

You should be familiar either with basic shell commands for Linux (ls, cp, mkdir ...).

You will need access to the SPEChpc 2021 installation media, typically as an ISO image or xz zipped tar package. The Installation Guides (Linux) explain how to use it.

On some systems, the mount command may require privileges or may require additional software. In such cases, you might need to burn a physical DVD using some other system; or, you might need to use the procedure described in the appendix to the Linux installation guide to extract a tarball and use that instead.

Please note that the SPEChpc 2021 license agreement does not allow you to post the SPEChpc 2021 software on any public server. If your institution has a SPEChpc 2021 license, then it's fine to post it on an internal server that is accessible only to members of your institution.

V. Don't be root. Turn privileges off. (Usually.)

Usually, you do not need privileges. The one known exception is that during installation, on some systems it is possible that you might need a privileged user to mount the installation media or to allocate resources, for example disk space.

After installation is complete, you should not need privileges to run SPEChpc benchmarks. In general, it is recommended that you use an ordinary user account for SPEChpc; that way, if your config file accidentally tries to delete the wrong directory, you are much less likely to damage your system.

Warning: SPEChpc config files can execute arbitrary shell commands.
Read a config file before using it.

SPEC recommends:

VI. Problems?

In case of difficulties, please check the document SPEChpc 2021 Frequently Asked Questions.

SPEChpc™2021 System Requirements: Copyright © 2021 Standard Performance Evaluation Corporation (SPEC)