---------------------------------------------------
NOTE: the following text file was automatically
generated from a document that is best read in
HTML format. To read it in the preferred format,
point your web browser at any of these 3 locations:
(1) http://www.spec.org/cpu2000/docs
(2) The docs directory of your SPEC CPU2000 CD -
for example:
/cdrom/docs/ (Unix)
E:\docs\ (NT)
(3) The docs directory on the system where you
have installed SPEC CPU2000 - for example
/usr/mydir/cpu2000/docs/ (Unix)
C:\My Directory$lcsuite\docs\ (NT)
---------------------------------------------------
SPEC CPU2000 Utility Programs
Last updated: 29-Oct-2001 jh
(To check for possible updates to this document, please see
http://www.spec.org/cpu2000/docs/
----------------------------------------------------------------------
Contents
* Introduction
* extract_config
* extract_raw
* go (Unix only)
* printkids.pl
* relocate (Unix only)
* rawformat
�1. What do you do when Marketing decides to change the name?
�2. About continuation fields in rawfiles
�3. How do you generate a 1-cpu rate result from a 1-cpu speed
result?
* specdiff
* specinvoke
* specmake
----------------------------------------------------------------------
Introduction
This document describes various useful utilities that ship with SPEC
CPU2000. It is assumed that you have already read runspec.html and
config.html.
-----------------------------------------------------------
Note: links to SPEC CPU2000 documents on this web page
assume that you are reading the page from a directory that
also contains the other SPEC CPU2000 documents. If by some
chance you are reading this web page from a location where
the links do not work, try accessing the referenced
documents at one of the following locations:
* www.spec.org/cpu2000/docs/
* The $SPEC/docs/ directory on a Unix system where SPEC
CPU2000 has been installed.
* The %spec%\docs.nt\ directory on a Windows/NT system
where SPEC CPU2000 has been installed.
* The docs/ or docs.nt\ directory on your SPEC CPU2000
distribution cdrom.
-------------------------------------------------------
----------------------------------------------------------------------
extract_config
If you have a rawfile, you can extract the config file that was used with
it. To do so, use the extract_config utility.
For example, starting with a freshly-installed SPEC CPU2000 directory
tree:
% runspec --config=Tru64_Unix --size=test --iterations=1 \
--tuning=peak --output_format=ps swim > /dev/null
% cd result
% ls
CFP2000.001.ps CFP2000.001.raw log.001 log.lock
Note that we have just the single result. Let's make a temporary directory
and copy the rawfile there:
% mkdir tmp
% cd tmp
% cp ../CFP2000.001.raw .
% ls
CFP2000.001.raw
Now, attempt to extract the config file:
% extract_config CFP2000.001.raw > tmp.cfg
Does the extracted file match the original?
% diff ../../config/Tru64_Unix.cfg ./tmp.cfg
0a1,3
> # Invocation command line:
> # /cpu2000/kit96/bin/runspec --config=Tru64_Unix --size=test
--iterations=1 --tuning=peak --output_format=ps swim
>
############################################################################
96,102d98
<
< __MD5__
< 171.swim=peak=oct6a=default:
< # Last updated Wed Dec 15 10:05:06 1999
< optmd5=764e8d466d967f5dacae93415dbfbe71
< exemd5=d7cccb73a236bbe7a548385ee4eff137
<
%
Yes. The only differences are:
* The copy in the rawfile adds a comment to tell you how the config file
was invoked.
* The original copy (in $SPEC/config) has the MD5 information for swim.
(The config file is tucked away into the rawfile prior to any
additions that are needed for MD5.)
NT Notes: on NT systems, you can invoke extract_config as:
specperl %spec%\bin\extract_config rawfile
----------------------------------------------------------------------
extract_raw
If you have a pdf, html, or ps output file, you can extract the rawfile
that was used to generate it. To do so, use extract_raw.
For example:
% ls
CFP2000.001.ps CFP2000.001.raw log.001 log.lock
% mkdir tmp
% cd tmp
% cp ../*ps .
% ls
CFP2000.001.ps
Note that at this point we have just the postscript output in the current
directory. Now, attempt to extract the raw file:
% extract_raw *ps
% ls
CFP2000.001.ps CFP2000.001.raw
Does the extracted file match the original?
% diff ../CFP2000.001.raw ./CFP2000.001.raw
%
Yes.
NT Notes: on NT systems, you can invoke extract_raw as:
specperl %spec%\bin\extract_raw file-to-extract-from
Note that the file you extract from can be pdf, ps, or html, just as on
Unix systems, with one important exception: if you are using a browser
that attempts to add value to the html when you save it, that browser may
break the embedded rawfile. For example, with Internet Explorer Version
5.00.3103.1000, if you are viewing a SPEC CPU2000 html file and you select
File -> Save As, the default for "Save as type" is shown as
"Web Page, complete (*.htm,*.html)"
Do not accept that default if you want to be able to retrieve the rawfile
later on. If you take the default, your saved file will be incomplete. To
save the web page INCLUDING the embedded rawfile, you must change "Save as
type" to
"Web Page, HTML only (*.htm,*.html)"
----------------------------------------------------------------------
go (Unix only)
When you source the shrc, as described in runspec.html, the 'go' shell
alias is created. Note: if you leave the shell where you sourced shrc, for
example by spawning csh, the alias is no longer available! So if you want
to use this alias, use a Bourne-compatible shell.
For those of you who will have it, here are the various things it
understands, as well as the places they'll take you. They're awfully
handy.
If you say You'll end up in
go $SPEC
go top $SPEC
go bin $SPEC/bin (where the tools live)
go config $SPEC/config
go result $SPEC/result
go results $SPEC/result
go doc $SPEC/docs
go docs $SPEC/docs
go int $SPEC/benchspec/CINT2000
go fp $SPEC/benchspec/CFP2000
go src foo foo's src directory, if it can be found. For
example:
go src 197 $SPEC/benchspec/CINT2000/197.parser/src
go src eon $SPEC/benchspec/CINT2000/252.eon/src
go src nada . (There's no benchmark named *nada*)
go work foo foo's run directory, if it can be found. For
example:
go work 197 $SPEC/benchspec/CINT2000/197.parser/run
go work eon $SPEC/benchspec/CINT2000/252.eon/run
go work nada . (There's no benchmark named *nada*)
go data foo foo's data directory, if it can be found. For
example:
go data 197 $SPEC/benchspec/CINT2000/197.parser/data
go data eon $SPEC/benchspec/CINT2000/252.eon/data
go data nada . (There's no benchmark named *nada*)
go foo foo's top level directory, if it can be found. For
example:
go 197 $SPEC/benchspec/CINT2000/197.parser
go eon $SPEC/benchspec/CINT2000/252.eon
go nada . (There's no benchmark named *nada*)
----------------------------------------------------------------------
printkids.pl
Some benchmark executables are invoked multiple times. If you'd like to
know how much time was spent on each invocation, it may not be easy to
tell. You could examine "speccmds.out", but that file is not easy to
interpret:
$ go work gcc
$ cat list | fold -s
00000001 dir=/cpu2000/kit92/benchspec/CINT2000/176.gcc/run/00000001
ext=nov11c_ns lock=0 mach=default size=ref tune=base type=run
username=john
__END__
$ cd *01
$
$ cat speccmds.out | fold -s
running commands in speccmds.cmd 1 times
runs started at 943371319, 399184000, Tue Nov 23 10:35:19 1999
run 1 started at 943371319, 399184000, Tue Nov 23 10:35:19 1999
child started: 0, 943371319, 400160000, 11128,
'../00000001/cc1_base.nov11c_ns
166.i -o 166.s'
child finished: 0, 943371519, 814960000, 200, 414800000, 11128, 0
child started: 0, 943371519, 815936000, 11135,
'../00000001/cc1_base.nov11c_ns
200.i -o 200.s'
child finished: 0, 943371854, 339648000, 334, 523712000, 11135, 0
child started: 0, 943371854, 339648000, 11154,
'../00000001/cc1_base.nov11c_ns
expr.i -o expr.s'
child finished: 0, 943371892, 803248000, 38, 463600000, 11154, 0
child started: 0, 943371892, 803248000, 11126,
'../00000001/cc1_base.nov11c_ns
integrate.i -o integrate.s'
child finished: 0, 943371939, 571936000, 46, 768688000, 11126, 0
child started: 0, 943371939, 571936000, 11146,
'../00000001/cc1_base.nov11c_ns
scilab.i -o scilab.s'
child finished: 0, 943372136, 993568000, 197, 421632000, 11146, 0
run 1 finished at: 943372136, 993568000, Tue Nov 23 10:48:56 1999
run 1 elapsed time: 817, 594384000, 817.594384000
runs finished at 943372136, 993568000, Tue Nov 23 10:48:56 1999
runs elapsed time: 817, 594384000, 817.594384000
Fortunately, SPEC has provided a tool to interpret the above for you:
$ printkids.pl
Seconds Command
200.41 cc1_base.nov11c_ns 166.i -o 166.s
334.52 cc1_base.nov11c_ns 200.i -o 200.s
38.46 cc1_base.nov11c_ns expr.i -o expr.s
46.77 cc1_base.nov11c_ns integrate.i -o integrate.s
197.42 cc1_base.nov11c_ns scilab.i -o scilab.s
=========
817.59 Total by adding children
817.59 Total according to speccmds.out
$
----------------------------------------------------------------------
relocate (Unix only)
If you rearrange mount points or rename your SPEC root directory, you will
probably find confusing problems like the one in the example that follows.
We used to have a working SPEC tree under /cpu2000/kit96, but we're going
to change that:
$ cd /cpu2000
$ mv kit96 larry
$ cd larry
$ . ./shrc
$ runspec -h
ksh: runspec: cannot execute
The problem is that the first line of runspec points to a path that no
longer exists:
$ head -1 bin/runspec
#!/cpu2000/kit96/bin/specperl
$
Fortunately, SPEC has provided a tool to relocate the tools in the new
tree:
$ bin/relocate
Top of SPEC tree is '/cpu2000/larry'
$ runspec -h | head -3
runspec v2.00 - Copyright (C) 1999 Standard Performance Evaluation
Corporation
Usage: bin/runspec [options]
LIMITATIONS: there are some limitations to this tool. Most notably, it
uses perl, so if the tree has been re-arranged in such a way that perl has
an identity crisis and can't find itself, you'll be in trouble. Make sure
that you have sourced shrc, as in the next example; if it still fails, you
may need to reinstall the CPU2000 kit.
$ mv larry carol
$ cd carol
$ bin/relocate
Top of SPEC tree is '/cpu2000/carol'
1062:/cpu2000/carol/bin/specperl: /sbin/loader: Fatal Error: cannot map
libperl.so
$ . ./shrc
$ bin/relocate
Top of SPEC tree is '/cpu2000/carol'
$ runspec -h | head -2
runspec v2.00 - Copyright (C) 1999 Standard Performance Evaluation
Corporation
Usage: bin/runspec [options]
$
----------------------------------------------------------------------
rawformat
"rawformat" is simply another name for
runspec --rawformat
so please see runspec.html for basic information on using this command.
But a couple of additional points are worth mentioning:
�1. What do you do when Marketing decides to change the name from
"SuperHero 4" to "SuperHero IV" just before you submit the results to
SPEC?
SPEC does allow you to edit the rawfile to make corrections. Just make
sure that you only edit the fields located before the "fence":
# =============== do not edit below this point ===================
WARNING: Make a backup copy of the rawfile first!
For example:
copy CFP2000.001.raw new.raw
(edit new.raw to make your changes)
rawformat -o ps new.raw
runspec v2.00 - Copyright (C) 1999 Standard Performance Evaluation
Corporation
Loading standard modules..............
Loading runspec modules.............
Identifying output formats...asc...config...html...pdf...ps...raw...
Formatting new.raw
format: PostScript -> new.ps
�2. About continuation fields in rawfiles
Notice when editting fields that have more than one line that you need
to construct the numbers with correct syntax. The syntax in the
rawfile is a bit more picky than in config files (compare the
following vs. the config.html section on "Field scoping and
continuation". You cannot say something like this:
WRONG:
spec.cpu2000.sw_compiler: Intel C Compiler 4.5 for Windows 2000
spec.cpu2000.sw_compiler1: Visual C++ 6.0 C (for libraries)
because the tools are not prepared at this point for a mixture of
numbered and un-numbered fields. If you need to have continued fields,
say instead:
RIGHT:
spec.cpu2000.sw_compiler0: Intel C Compiler 4.5 for Windows 2000
spec.cpu2000.sw_compiler1: Visual C++ 6.0 C (for libraries)
�3. How do you generate a 1-cpu rate result from a 1-cpu speed result?
To generate a 1-cpu rate result from a speed run, copy the original
rawfile to another location, and use rawformat to both generate the
new rawfile and whatever other reports you want. For example:
$ grep SPECf CFP2000.015.asc
SPECfp_base2000 176
SPECfp2000 --
$ cp CFP2000.015.raw convertme
$ rawformat --output_format asc,raw,ps --rate convertme
runspec v2.00 - Copyright (C) 1999 Standard Performance Evaluation Corporation
Loading standard modules..............
Loading runspec modules.............
Identifying output formats...asc...config...html...pdf...ps...raw...
Formatting convertme
format: ASCII -> convertme.asc
format: raw -> convertme.raw
format: PostScript -> convertme.ps
$ grep SPECf convertme.asc
SPECfp_rate_base2000 2.05
SPECfp_rate2000 --
NT Notes: on NT systems, you may find that --output_format will only
accept one argument at a time. So, first create the rawfile, by using
--output_format raw, then use the new rawfile to create the other
reports.
----------------------------------------------------------------------
specdiff
In order to decide whether your system got the correct answer for a
benchmark, runspec writes a file called "compare.cmd" in the run
directory, and then calls specinvoke, which calls specdiff. (The
specinvoke utility is described later in this document.)
Suppose that your results are flagged as incorrect, but you'd like to have
more detail. You could ask for more than 10 lines of detail by changing
the config file's default for difflines from 10 to something else, and
re-running the experiment. But, you might save a lot of time and avoid the
re-run of the experiment, by calling specdiff directly, with a different
number of lines for its switch -l.
For example, a file chock-full of wrong answers for ammp has been created
by hand. First, let's see how specdiff would normally be run (line wraps
added for readability):
$ specinvoke -c 1 -f compare.cmd -n
/cpu2000/kit96/bin/specdiff -m -l 10 -a 0.0001 -r 0.003
/cpu2000/kit96/benchspec/CFP2000/188.ammp/data/test/output/ammp.out
ammp.out > ammp.out.cmp
The above says that normally we test for correct answers by comparing
data/test/output/ammp.out to ammp.out in the current directory. Normally,
10 lines of detail are printed. Let's change that to 15 lines, and compare
versus our intentionally wrong file instead:
$ /cpu2000/kit96/bin/specdiff -m -l 15 -a 0.0001 -r 0.003 \
/cpu2000/kit96/benchspec/CFP2000/188.ammp/data/test/output/ammp.out \
ammp.wrong_output
0002: setf mxdq 0.75;
setf mxdq 01.75;
^
0003: setf mmbox 10.;
setf mmbox 101.;
^
0006: setf bbox 35.;
setf bbox 351.;
^
0009: 0.000000 unknown potential type
01.000000 unknown potential type
^
0010: 1181.693079 bond energy
11811.693079 bond energy
^
0011: 2056.767980 angle energy
20561.767980 angle energy
^
0012: 61.501074 hybrid energy
611.501074 hybrid energy
^
0013: 533.960821 torsion energy
5331.960821 torsion energy
^
0014: -15410.005122 non-bonded energy
-154101.005122 non-bonded energy
^
0015: -11576.082168 total potential energy
-115761.082168 total potential energy
^
0016: 8584.512662 total kinetic energy
85841.512662 total kinetic energy
^
0017: -2991.569506 total energy
-29911.569506 total energy
^
0018: 20160.594829 total action
201601.594829 total action
^
0020: tpac numstp .00001 temp;
tpac numstp 1.00001 temp;
^
0022: 0.000000 unknown potential type
01.000000 unknown potential type
^
0023: 1184.181101 bond energy
11841.181101 bond energy
^
$
Here are the switches supported by specdiff:
$ specdiff -h
Usage: /cpu2000/kit96/bin/specdiff [-l #] [-q] file1 [file2]
-l # of lines of differences to print
-q don't print lines just set return code
-a absolute tolerance (for floating point compares)
-r relative tolerance (for floating point compares)
-t set calctol
-s set skiptol
-o allow off-by-one errors
-O *don't* allow off-by-one errors
-m write file2.mis with miscompares
-c collapse whitespace (doesn't do what you think it does)
-C *don't* collapse whitespace
-d Compare against file(s) in data directory
--os Set the operating system type (you don't need to do this)
-v Set the level of noisiness for the output
-h Print this message
$
----------------------------------------------------------------------
specinvoke
In order to run a benchmark, runspec writes a series of commands in the
run directory, in the file "speccmds.cmd". These commands are then
interpreted by specinvoke. When something goes wrong, it is often useful
to try the commands by hand. Ask specinvoke what it did, by using the "-n"
switch:
$ go work swim
$ cd 00000002
$ ls
SWIM7 compare.out swim.err swim_peak.oct6a
SWIM7.cmp speccmds.cmd swim.in
compare.cmd speccmds.err swim.out
compare.err speccmds.out swim.out.cmp
$ specinvoke -n
../00000002/swim_peak.oct6a swim.out 2> swim.err
$
In the above example, we see how swim is invoked. If you wanted to enter
your own variations on the above command by hand, you could do so. Here
are the switches supported by specinvoke:
$ specinvoke -h
Usage: specinvoke [options] [instructionfile]
-i # iterations
-c # concurrent processes (overrides -u in command file)
-E return non-zero exit code if child does
-s shell shell to invoke
-f file instruction file
-o file output file
-e file error file
-d dir change to dir first
-n[n] print a 'dry_run' of commands
-A return 0 error code
-r don't do command I/O redirection ($command already has it)
-h this message
----------------------------------------------------------------------
specmake
To build a benchmark, runspec uses specmake, which is simply gnu make
under a unique name to avoid possible conflicts with other versions of
make on your system. If a benchmark build fails, the most massively useful
thing you can do is to ask specmake what it did, by using the "-n" switch
for dry run. For example, let's create a config file that is unlikely to
work:
$ cat >tmp.cfg
ext = silly
FC = f66
$
and try it out:
$ runspec --config=tmp --tuning=base --action=build swim
runspec v2.00 - Copyright (C) 1999 Standard Performance Evaluation
Corporation
Loading standard modules..............
Loading runspec modules.............
Identifying output formats...asc...config...html...pdf...ps...raw...
We will use: 171.swim
Compiling Binaries
Building 171.swim ref base silly default
Error with make 'specmake build > make.out 2> make.err': check file
'/cpu2000/kit96/benchspec/CFP2000/171.swim/run/00000004/make.err'
Error with make!
*** Error building 171.swim
If you wish to ignore this error, please use '-I' or ignore errors.
$
Sure enough, it failed. Let's find the appropriate run directory where our
silly build was attempted:
$ go run swim
$ grep build list | grep silly | cut -b 1-9
00000004
$ cd 00000004
And ask specmake just exactly what it generated:
$ specmake -n
f66 -c -o swim.o swim.f
f66 swim.o -o swim
OK, let's enter the command by hand:
$ f66 -c -o swim.o swim.f
ksh: f66: not found
$
A great convenience of the CPU2000 suite is the fact that you can now
proceed to play with all the files in this run directory, to prototype
different changes and try out ideas, without fear that you will break
anything. As long as you confine your edits to (in this example)
$SPEC/benchspec/CFP2000/171.swim/run/00000004/, you are in a private
sandbox and will not harm the original sources.
Here are the switches supported by specmake:
$ specmake -h
Usage: specmake [options] [target] ...
Options:
-b, -m Ignored for compatibility.
-C DIRECTORY, --directory=DIRECTORY
Change to DIRECTORY before doing anything.
-d, --debug Print lots of debugging information.
-e, --environment-overrides
Environment variables override makefiles.
-f FILE, --file=FILE, --makefile=FILE
Read FILE as a makefile.
-h, --help Print this message and exit.
-i, --ignore-errors Ignore errors from commands.
-I DIRECTORY, --include-dir=DIRECTORY
Search DIRECTORY for included makefiles.
-j [N], --jobs[=N] Allow N jobs at once; infinite jobs with no arg.
-k, --keep-going Keep going when some targets can't be made.
-l [N], --load-average[=N], --max-load[=N]
Don't start multiple jobs unless load is below N.
-n, --just-print, --dry-run, --recon
Don't actually run any commands; just print them.
-o FILE, --old-file=FILE, --assume-old=FILE
Consider FILE to be very old and don't remake it.
-p, --print-data-base Print make's internal database.
-q, --question Run no commands; exit status says if up to date.
-r, --no-builtin-rules Disable the built-in implicit rules.
-s, --silent, --quiet Don't echo commands.
-S, --no-keep-going, --stop
Turns off -k.
-t, --touch Touch targets instead of remaking them.
-v, --version Print the version number of make and exit.
-w, --print-directory Print the current directory.
--no-print-directory Turn off -w, even if it was turned on implicitly.
-W FILE, --what-if=FILE, --new-file=FILE, --assume-new=FILE
Consider FILE to be infinitely new.
--warn-undefined-variables Warn when an undefined variable is referenced.
----------------------------------------------------------------------
Copyright (C) 1999-2000 Standard Performance Evaluation Corporation
All Rights Reserved