---------------------------------------------------
NOTE: the following text file was automatically
generated from a document that is best read in
HTML format. To read it in the preferred format,
point your web browser at any of these 3 locations:
(1) http://www.spec.org/cpu2000/docs
(2) The docs directory of your SPEC CPU2000 CD -
for example:
/cdrom/docs/ (Unix)
E:\docs\ (NT)
(3) The docs directory on the system where you
have installed SPEC CPU2000 - for example
/usr/mydir/cpu2000/docs/ (Unix)
C:\My Directory$lcsuite\docs\ (NT)
---------------------------------------------------
SPEC CPU2000: The runspec Command
Last updated: 18-Oct-2001 jh
(To check for possible updates to this document, please see
http://www.spec.org/cpu2000/docs/ )
Table of Contents
I. Introduction
A. Who Needs runspec?
B. About Config Files
C. About Defaults
D. About Disk Usage and Support for Multiple Users
II. Before Using runspec
Install kit
Have a config file
Clean the environment
Set your path: Unix
Set your path: NT
Check your disk space
III. Using runspec
A. Simplest usage
1. Reportable run
2. Running selected benchmarks
3. Output files
B. Syntax
C. Actions
D. Commonly used options
--action --config --help --ignore_errors
--iterations --loose --output_format --rate
--rawformat --rebuild --reportable --tune
--users
E. Less commonly used options
--deletework --extension --machine
--make_no_clobber --maxcompares --[no]setprocgroup
--size --[no]table --unbuffer --username
--verbose --version
IV. Quick reference
----------------------------------------------------------------------
I. Introduction
A. Who Needs runspec?
Everyone who uses SPEC CPU2000 needs runspec. It is the primary tool in
the suite. It is used to build the benchmarks, run them, and report on
their results. All users of CPU2000 should read this document.
B. About Config Files
In order to use runspec, you need a "config file", which contains detailed
instructions about how to build and run the benchmarks. You may not have
to learn much about config files in order to get started. Typically, you
start off using a config file that someone else has previously written.
Where can you find such a config file? There are 2 primary sources:
�1. Look in directory $SPEC/config (Unix) or %SPEC%\config (NT). You may
find that there is a already a config file there with a name that
indicates that it is appropriate for your system. You may even find
that default.cfg already contains settings that would be a good
starting place for your system.
�2. Look at the SPEC web site (http://www.spec.org/cpu2000/) for a CPU2000
result submission that used your system, or a system similar to yours.
You can download the config file from that submission.
Alternatively, you can write your own, using the instructions in
config.html
-----------------------------------------------------------
Note: links to SPEC CPU2000 documents on this web page
assume that you are reading the page from a directory that
also contains the other SPEC CPU2000 documents. If by some
chance you are reading this web page from a location where
the links do not work, try accessing the referenced
documents at one of the following locations:
* www.spec.org/cpu2000/docs/
* The $SPEC/docs/ directory on a Unix system where SPEC
CPU2000 has been installed.
* The %spec%\docs.nt\ directory on a Wind ows/NT system
where SPEC CPU2000 has been installed.
* The docs/ or docs.nt\ directory on your S PEC CPU2000
distribution cdrom.
-------------------------------------------------------
See also the examples in
$SPEC/docs/*.cfg (Unix) or
%SPEC%\docs.nt\*.cfg (NT).
C. About Defaults
The SPEC tools have followed two design principles regarding defaults:
�1. There should always be a default for everything
�2. It should be easy to change the defaults
This means (the good news) that something sensible will usually happen,
even when you aren't explicit about what you want. But it also means (the
bad news) that if something unexpected happens, you may have to look in
several places in order to figure out why it behaves differently than you
expect.
The order of precedence for settings is:
Highest precedence: runspec command
Middle: config file
Lowest: the tools as shipped by SPEC
Therefore, when this document tells you that something is the default,
bear in mind that your config file may have changed that setting. If in
doubt, check the config file.
D. About Disk Usage and Support for Multiple Users
The structure of the CPU2000 directory tree is:
$SPEC or %SPEC% - the root directory
benchspec
CFP2000 - floating point benchmarks
CINT2000 - integer benchmarks
bin - tools to run and report on the suite
config - config files
result - log files and reports
tools - sources for the CPU2000 tools
Within each of the individual benchmarks, the structure is:
nnn.benchmark - root for this benchmark
Spec - SPEC metadata about the benchmark
data
all - data used by all runs (if needed by the benchmark)
ref - the real data set, required for all result reporting
test - data for a simple test that an executable is functional
train - data for feedback-directed optimization
exe - compiled versions of the benchmark
run - all builds and runs take place here
src - the sources for the benchmark
When you find yourself wondering "Where did all my disk space go?", the
answer is "The run directories." All build and run activity takes place in
numbered run directories - e.g.
$SPEC/benchspec/CINT2000/164.gzip/run/00000001
To get your disk space back, see the documentation of the various cleaning
options, below; or issue a command such as the following (on Unix systems;
NT users can select the files with Explorer):
rm -Rf $SPEC/benchspec/C*/*/run
Because all the build and run activity takes place in separately created
directories, the tools are able to provide limited support for multiple
users of the same directory tree:
* Each numbered run directory is tagged with the name of the user that
it belongs to (in the file nnn.benchmark/run/list).
* Multiple jobs can be submitted at the same time by one or more users.
* Different users can be doing simultaneous builds in different run
subdirectories, or even simultaneous runs. (Of course, if multiple
jobs compete with each other for resources, it is likely that they
will run more slowly.)
The support for multiple users is subject to some limitations:
�1. The directory tree must be writable by each of the users, which means
that the users have to trust each other not to modify or delete each
others' files.
�2. Directories such as config, result, and exe are not segregated by
user, so you can only have one version of (for example)
config/phil.cfg or benchspec/CINT2000/164.gzip/exe/gzip_base.les
�3. Support for multiple users has received only limited testing during
the development of CPU2000, and may have bugs. If you encounter a
problem, please report it to SPEC.
If these limitations cause concern, you should give each user their own
copy of the entire SPEC CPU2000 directory tree, and use your operating
system's file protection scheme.
----------------------------------------------------------------------
II. Before Using runspec
Before using runspec, you need to:
�1. Successfully install CPU2000. Runspec uses perl version 5.005_03,
which is installed as specperl when you install CPU2000.
Please see install_guide_unix.html or install_guide_nt.html.
�2. Find a config file. You won't get anywhere unless you have a config
file, but fortunately you can get started by using a pre-existing
config file. See About Config Files, above.
�3. Make sure that the environment variable SPEC is NOT defined when you
start out. If it is already defined (e.g. from a run of some other
SPEC benchmark suite), you should undefine it first, e.g. by logging
out and logging in, or by using unset.
To check whether the variable is already defined, type
echo $SPEC (Unix)
or
echo %SPEC% (NT)
On Unix systems, the desired output is nothing at all; on NT systems,
the desired output is %SPEC%.
�4. Set your path appropriately for your system type (Unix or NT/Windows
2000)
a. If you are using a Unix system:
Change your current directory to the top-level SPEC directory and
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!! START A BOURNE-COMPATIBLE SHELL !!!!!!!!!!!!!
! (if you are not already in one) !
! !
! That is, you need to use a command interpreter that !
! supports the syntax used by shrc, which tries to be !
! highly portable by avoiding proprietary or uncommon !
! shell features. Many shells -- such as ash, bash, !
! ksh, and zsh -- provide some degree of compatibility !
! with the Bourne shell. The C shell (csh) definitely !
! is NOT compatible with Bourne shell syntax. If you !
! are in doubt, or if you see unexpected error messages !
! from shrc, you can usually start a Bourne shell by !
! typing 'sh'. !
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Then, source the file shrc. If you prefer some other shell, you can
probably spawn it after sourcing shrc; when you do so, please be sure
that your new shell does not clear the path variables set up by shrc!
For example:
% cd /usr/rahul/cpu2000
% sh
$ . ./shrc <- that's dot-space-dot-slash-shrc
$ csh
%
b. If you are using a Windows/NT system:
Start a command window, then type:
notepad shrc.bat
and make the appropriate edits for your compiler paths.
Caution: you may find that the lines are not correctly formatted (the
text appears to be all run together) when you edit this file. If so,
see the section "Using Text Files on NT" in the NT installation guide.
Then set the path using your edited shrc.bat, for example:
F:\bodo\cpu2000> shrc
�5. Make sure that you have enough disk space. Typically, you will want to
have at least 1/2 GB free disk space at the start of a run. Please see
About Disk Usage and Support for Multiple Users, above.
----------------------------------------------------------------------
III. Using runspec
A. Simplest usage
1. Reportable run
Runspec is easiest to use when:
* Some kind person has already compiled the benchmarks.
* That kind person provides both the compiled images and their
corresponding config file (see About Config Files above).
* The config file does not change the defaults in surprising or esoteric
ways (see About Defaults above).
In this lucky circumstance, all that needs to be done is to name the
config file, select which benchmark suite is to be run: int for CINT2000
or fp for CFP2000, and say --reportable if one wants to attempt a full run
that complies with the SPEC CPU2000 run rules.
For example, suppose that Wilfried has given Ryan a config file with some
new integer optimizations for a Unix system. Ryan could say:
% cd /usr/ryan/cpu2000
% sh
$ . ./shrc
$ runspec --config wilfried_new.cfg --reportable int
As a another example, suppose that Reinhold has given Kaivalya an NT
config file with changes from 12 August, and Kaivalya wants to run the
floating point suite. He might say something like this:
F:\kaivalya\cpu2000> shrc
F:\kaivalya\cpu2000> runspec --config reinhold_aug12a --reportable fp
2. Running selected benchmarks
If you want to run a subset of the benchmarks, rather than running the
whole suite, you can name them. Since a reportable run uses an entire
suite, you'll need to turn off reportable:
% runspec --config arrian_dec25j.cfg --noreportable 252.eon 181.mcf
3. Output files
Look for the output of your runspec command in the directory $SPEC/result
(Unix) or %SPEC%\result (NT). There, you will find log files and result
files. More information about log files can be found in config.html.
The format of the result files depends on what was selected in your config
file, but will typically include at least .asc for ASCII text, and will
always include .raw, for raw (unformatted) run data. More information
about result formats can be found below, under --output_format. Note that
you can always re-generate the output, using the --rawformat option, also
documented below.
.........................................
This concludes the section on simplest usage;
if simple commands such as the above don't seem
to do what you like, then you'll have to keep
reading this document.
.........................................
B. Syntax
The syntax for the runspec command is:
runspec [options] [list of benchmarks to run]
Options are described in the following sections. There, you you will
notice that many options have both long and short names. The long names
are invoked using two dashes, and the short names use only a single dash.
For long names that take a parameter, you can optionally use an equals
sign. Long names can also be abbreviated, provided that you still enter
enough letters for uniqueness. For example, the following commands all do
the same thing:
runspec --config=dianne_july25a --debug=99 fp
runspec --config dianne_july25a --debug 99 fp
runspec --conf dianne_july25a --deb 99 fp
runspec -c dianne_july25a -v 99 fp
The list of benchmarks to run can be either an entire suite ("int" or
"fp") or one or more individual benchmarks. Individual benchmarks can be
named, numbered, or both; and they can be abbreviated, as long as you
enter enough characters for uniqueness. For example, each of the following
commands does the same thing:
runspec -c jason_july09d --noreportable 187.facerec 168.wupwise
runspec -c jason_july09d --noreportable 187 168
runspec -c jason_july09d --noreportable facerec wupwise
runspec -c jason_july09d --noreportable face wup
It is also possible to exclude a benchmark, using a hat (^, also known as
carat, typically found as shift-6). For example, suppose your system lacks
a C++ compiler, and you therefore cannot run the benchmark 252.eon. You
could run all of the integer benchmarks except eon by entering a command
such as this one:
runspec --noreportable -c kathy_sep14c int ^eon
Note that if hat has significance to your shell, you may need to protect
it from interpretation by the shell, for example by putting it in single
quotes. On NT, you will need to use both a hat and two quotes for each
benchmark you want to exclude, like this:
E:\cpu2000> runspec --noreportable -c debbie_apr26a int "^gzip" "^eon"
C. Actions
Every time runspec is used, it normally takes some kind of action for the
set of benchmarks specified at the end of the command line (or defaulted
from the config file). The default action is validate, which means that
the benchmarks will be built if necessary, the run directories will be
setup, the benchmarks will be run, and reports will be generated.
If you want to force a different action, then you can enter one of the
following runspec options:
--action build Compile the benchmarks. More information about
compiling may be found in config.html, including
information about additional files that are output
during a build.
--action run Run the benchmarks but do not bother to verify that
they got the correct answers. This option is not
normally useful, but can be selected if, for example,
you are generating a performance trace and wish to
avoid tracing some of the tools overhead.
--action setup Setup the run directories. Copy executables and data to
work directories.
--action validate Build (if needed), run, and generate reports
In addition, the following cleanup actions are available (in order by
level of vigor):
--action clean Empty all run and build directories for the specified
benchmark set. For example, if user Jeff enters the
command: runspec --action clean --config may12a fp the
tools would remove run directories for the floating
point benchmarks which are tagged as belonging to Jeff
and which contain benchmarks generated by config file
may12a.cfg.
--action trash Same as clean, but do it for all users of this SPEC
directory tree.
--action realclean A synonym for --action trash
--action clobber Clean + remove all executables of the current type
--action nuke Remove everybody's run and build directories and all
executables
On the other hand, if you prefer to do all cleaning by hand (perhaps after
reviewing the results of each run), you can ask the tools to never touch a
used run directory. Do this by setting the environment variable:
SPEC_CPU2000_NO_RUNDIR_NUKE
Alternative cleaning method:
If you prefer, you can clean disk space by entering commands such as the
following (on Unix systems):
rm -Rf $SPEC/benchspec/C*/*/run
rm -Rf $SPEC/benchspec/C*/*/exe
NT users can achieve a similar effect using Windows NT Explorer. Notice
that the above commands not only empty the contents of the run and exe
directories; they also delete the directories themselves. That's fine; the
tools will re-create the run and exe directories if they are needed again
later on.
D. Commonly used options
Most users of runspec will want to become familiar with the following
options.
--action action
* Synonym: -a action
* Default: validate
* Meaning: One of the following actions that runspec can do for you:
build clean clobber nuke run setup trash or validate. For more
information, see Actions, above
--config name
* Synonym: -c name
* Default: default.cfg
* Meaning: Use config file config/name.cfg. If not specified, the tools
will try config/default.cfg
--help
* Synonyms: -h, -?
* Default: Help text is not displayed.
* Meaning: Print helpful text
--ignore_errors
* Synonyms: -I, --ignoreerror
* Default: Stop on first error.
* Meaning: Continue running even if errors occur. Note that a run with
--ignore_errors cannot be submitted to SPEC.
--iterations number
* Synonym: -n number
* Default: 3
* Meaning: How many times to run each benchmark. For a reportable run,
must be an odd number >= 3.
--loose
* Synonyms: -l, --nostrict, --noreportable
* Default: loose
* Meaning: Unset reportable. For example, suppose that you have a golden
config file that is used for reportable runs, and which sets the
config file option reportable. But you aren't ready to do a reportable
run, because you first want to ensure that the system under test is
adequately configured to run the benchmark 252.eon. Since the config
file sets the option reportable, the following command will result in
an error message:
runspec --config golden --iterations 1 eon
as the SPEC tools will inform you that you cannot change the number of
iterations on a reportable run. But either of the following commands
will override the config file and just run eon once:
runspec --config golden --iterations 1 --loose eon
runspec --config golden --iterations 1 --noreportable eon
--output_format format
* Synonym: -o format
* Default: all available formats
* Meaning: Desired report format, one or more of
* all - all of the following
* asc - ASCII text
* config - config file used for this run
* html - web page
* pdf - Portable Document Format
* ps - Postscript
If more than one option is used, separate them by commas.
--rate
* Synonym: -r
* Default: speed run (i.e. non-rate)
* Meaning: Select rate run instead of speed. Normally you would also
specify --users. For example, the following command might be used for
a 4-user SPECint_rate2000 run:
runspec --config ken_may12a --rate --users 4 int
If you have also entered --rawformat, then the effect of --rate is to
format the rawfile for rate even if it was originally a speed run.
That is, it is valid to report a single user run as both speed and
rate, but only for the case where you first run it as a speed run, and
then reformat for rate.
--rawformat rawfiles
* Synonym: -R rawfile
* Default: Do not use rawformat mode.
* Meaning: Take an existing run and just generate the reports. This
option is useful if (for example) you are just doing ASCII output
during most of your runs, but now you would like to create additional
reports for one or more especially interesting runs. To create the
html and postscript files for experiment number 324, you could say:
runspec --rawformat --output_format html,ps \
result/CINT2000.324.raw
You can achieve the same effect by using rawformat:
rawformat --output_format html,ps \
result/CINT2000.324.raw
For more information about rawformat, please see utility.html
--rebuild
* Synonym: -D
* Default: Do not rebuild unless binary is missing or does not match the
config file.
* Meaning: Delete existing binaries and recompile. Use this if, for
example, you have installed a new version of your compiler and want to
force a rebuild of the benchmarks.
--reportable
* Synonyms: -s, --strict, --noloose
* Default: loose
* Meaning: Wherever it is practical to do so in an automated fashion,
enforce the CPU2000 run rules, so as to produce a result which is
suitable for public reporting and/or submission to SPEC. Note that
this option forces various other options, such as iterations odd and
>=3.
--tune tuning
* Synonyms: --tuning tuning, -T tuning
* Default: base
* Meaning: Selects tuning to use: base, peak, or all. For a reportable
run, must be either base or all. Reportable runs always run base
first, then peak.
--users number
* Synonyms: -u number
* Default: 1
* Meaning: Use number copies for a SPECrate run. See --rate.
E. Less commonly used options
--deletework
* Synonym: -d
* Default: Do not delete working directories.
* Meaning: Delete run directories and re-populate. Use this if you
suspect that the run directories may have been corrupted, for example
due to a system crash during setup of an earlier run. You might also
use this if you just wish to ensure clean run directories, but note
that the tools will automatically provide what they deem to be a
sufficient level of cleaning.
--extension name
* Synonyms: --ext name, -e name
* Default: none
* Meaning: Extension to build or run. Normally used only if the config
file has been written to handle more than one extension. The config
file author should tell you what extensions to enter under which
circumstances.
--machine name
* Synonym: --mach name, -m name
* Default: none
* Meaning: The machine to build for or to run. Normally used only if the
config file has been written to handle more than one machine type. The
config file author should tell you what machines are supported by the
config file.
--make_no_clobber
* Synonym: -C
* Default: clobber
* Meaning: Do not delete existing object files before attempting to
build. This option should only be used for troubleshooting a
problematic compile. It is against the run rules to use it when
building binaries for an actual submission.
For a better way of troubleshooting a problematic compile, see the
information about specmake in utility.html
--maxcompares N
* Synonym: --max_active_compares N
* Default: the same as number specified for --users
* Meaning: Set the number of concurrent compares to N. This option is
useful if you have multiple CPUs and are doing a rate run. Do not
exceed the value that was specified for --users.
--setprocgroup, --nosetprocgroup
* Synonyms: none
* Default: --setprocgroup
* Meaning: Attempt to create all processes in a single group. Improves
the chances that ^C will get the whole run, not just one of the
children.
--size size
* Synonyms: -i size, --input size
* Default: ref
* Meaning: Selects size of input data to run: test, train, or ref.
Reportable runs always use ref. You might choose to use a smaller data
size while debugging a set of compilation options.
--table, --notable
* Synonyms: none
* Default: --table
* Meaning: For ASCII reports, include a detailed table of all runs
--unbuffer
* Synonym: -f
* Default: buffer
* Meaning: Do not buffer output. For SPEC CPU95, there were occasional
complaints about redundant output when a child process would flush its
buffer. If similar problems occur for CPU2000, try --unbuffer.
--username name
* Synonym: -U name
* Default: same as current process
* Meaning: Name of user. Useful if multiple users are sharing a SPEC
tree. Please see "About Disk Usage and Support for Multiple Users",
above.
--verbose n
* Synonyms: --debug n, -v n
* Default: 5
* Meaning: Set verbosity level to a level between 1 and 99. For more
information, see config.html
--version
* Synonym: -V
* Default: Print short version information.
* Meaning: Print detailed version information for the SPEC CPU tool
suite.
----------------------------------------------------------------------
IV. Quick reference
-a Same as --action
--action action Do: build clean clobber nuke run setup trash or
validate
-C Same as --make_no_clobber
-c Same as --config
--config file Set config file for runspec to use
-D Same as --rebuild
-d Same as --deletework
--debug Same as --verbose
--deletework Force work directories to be rebuilt
-e Same as --extension
--ext Same as --extension
--extension ext Set the extension
-f Same as --unbuffer
-h Same as --help
--help Print usage message
-I Same as --ignore_errors
-i Same as --size
--ignore_errors Continue with benchmark runs even if some fail
--ignoreerror Same as --ignore_errors
--input Same as --size
--iterations N Run each benchmark N times
-l Same as --loose
--loose Do not produce a reportable result
--noloose Same as --reportable
-m Same as --machine
--mach Same as --machine
--machine name Set the machine type
--make_no_clobber Do not delete existing object files before
building.
--max_active_compares Same as --maxcompares
--maxcompares N Set the number of concurrent compares to N
-n Same as --iterations
--noreportable Same as --loose
-o Same as --output_format
--output_format format Generate report, one or more of: asc html pdf ps
config
-R Same as --rawformat
-r Same as --rate
--rate Do a throughput (rate) run
--rawformat Format raw file
--rebuild Force a rebuild of binaries
--reportable Produce a reportable result
-s Same as --reportable
--[no]setprocgroup [Don't] try to create all processes in one group.
--size Select data set, one of: test train ref
--strict Same as --reportable
--nostrict Same as --loose
-T Same as --tune
--[no]table Do [not] include a detailed table of results
--tune Set the tuning levels to one of: base peak all
--tuning Same as --tune
-u Same as --users
--users Set the number of users for a rate run
-U Same as --username
--unbuffer Turn output buffering off
--username Name of user to tag as owner for run directories
-v Same as --verbose
--verbose Set verbosity level for messages to N
-V Same as --version
--version Output lots of version information
-? Same as --help
----------------------------------------------------------------------
Copyright (C) 1999-2001 Standard Performance Evaluation Corporation All
Rights Reserved