GCC Benchmark Specifications 1. GENERAL 1.1. Classification gcc is a CPU intensive integer benchmark written in C. It spends 5% to 10% of its total execution time performing disk I/O on a Sun-4. About 0.015% of all instructions executed on a Sun-4 are floating point. 1.2 Description gcc is based on the GNU C compiler version 1.35 distributed by the Free Software Foundation. The benchmark measures the time it takes for the GNU C compiler to convert a number of its pre-processed source files into optimized Sun-3 assembly language (.s files) output. 1.3 Source/Author The gcc benchmark is sponsored by Sun Microsystems Inc.. It is derived from the GNU C compiler written principally by Richard Stallman of the Free Software Foundation. 1.4 Version/Date Benchmark Version 1.0, Date 5/18/89. 1.5 Other Information The benchmark unloads from tape into a directory called 085.gcc. This benchmark has been tested on a Sun-4, DECstation 3100, MIPS M/2000, Apollo Domain 10000 and an HP 835 machine. Please email nvc@sun.com to report problems or to request further information. 2. PERFORMANCE 2.1. Metrics The metric used is the elapsed (real) time as output by /bin/time. This is the time that the GNU C compiler takes to compile 19 pre-processed source files into assembly files. 2.2. Elapsed Time As above. The SPEC reference time (to 3 sig. fig.) is 5460 seconds. 2.3. Reports The output of /bin/time is directed to result/time.out. The benchmark creates a number of assembly language output files. 2 These output files are compared with a set of reference files. 3 2.4 Additional Performance Considerations gcc is mainly an integer benchmark. However, it does contain a small amount of I/O and floating point computation. On a Sun-4/260 system the disks are busy about 8% of the total run time of the benchmark. About 0.01% of the instructions on a Sun-4/260 are floating point.The benchmark should be run 5 times by default to help eliminate distortions due to vari- ance. The arithmetic mean of the elapsed time of all the runs should be reported as the performance metric. The vari- ance in execution times is much lower than 5% among succes- sive runs of the benchmark. We recommend that the benchmark be run on a system in single user mode or one with a low load average to ensure repeatability of results. 3. SOFTWARE 3.1 Language The gcc benchmark is written in C. 3.2 Operating System The benchmark has been run on Unix System V and BSD 4.3 based operating systems. 3.3 Portability Installation of the GNU C compiler is system dependent. We have minimized the amount of reconfiguration required to run the benchmark on different machines and operating systems. The native compilers on some machines may not be able to com- pile the GNU C compiler without a change in the source code. We request that these changes be propagated back to nvc@sun.com. The machines we tested the benchmark on com- piled the GNU C compiler with some warning messages. Exam- ples of the output of the compilation may be found in the files gcc1.35/Make.log.*. 3.4 Vectorizability/Multi-Processor Issues gcc code is not easily vectorizable. 3.5 Miscellaneous Software The benchmark requires the presence of the Unix utilities cat, cp, diff, ln, rm, mv, sh and time. 4 3.6 Known Bugs Since this benchmark is highly system dependent, it may fail on some machines. We have attempted to fix most problems, however, not all warning messages have been suppressed. 3.7 Additional Software Considerations None. 5 4. HARDWARE 4.1 Memory The dynamic memory requirements of this benchmark have not been ascertained. We have found that 8MB systems tend to do some paging. There is insignificant paging on 16MB systems. The static size of the executable modules gcc and cc1 on a Sun-4 are as follows: hostname% size gcc cc1 text data bss dec hex 40960 8192 408 49560 c198 gcc 638976 90112 15456 744544 b5c60 cc1 4.2 Disks No special requirements. 4.3 Communication None. 4.4 Special Hardware None. 4.5 Additional Hardware Considerations None 5. OPERATIONAL 5.1 Disk Space gcc takes up about 8.5 MB of space on tape. The source and executables require under 15 MB of disk space. hostname% du 1438 ./config # config files for different machines 1283 ./gcc.txt # Latex files containing gcc info 929 ./input.ref # Pre-processed input files 737 ./result.ref # Reference assembly language output files 8473 . # gcc source 6 5.2 Installation Extract using tar. There is an extensive README and a generic Makefile called gcc1.35/Makefile. We have also included a sample makefile for the Sun-4. At present, the only system specific configuration activity required is to link gcc1.35/config.h to gcc1.35/config/xm-m68k.h for Unix 4.3 BSD based systems (default setting) or to gcc1.35/config/xm- hp9k320.h for Unix System V systems. While this has worked on the machines we have tested, there may be some machines and operating systems that will require more work. The user should read the file gcc1.35/INSTALL in such situations. 5.3 Execution To execute the benchmark type the following (substitute the appropriate name in place of ): hostname% cd gcc1.35 hostname% make -f M. run The makefile will compile the benchmark if necessary and then run it. It will create a directory called result and store the timing output in a file called time.out. The benchmark will also create a number of Sun-3 assembly language (.s) files in the result directory. If the target run is elim- inated from the make command-line arguement above, the make command will compile, run and validate the benchmark. 5.4 Correctness Verification To verify the benchmark type: hostname% make -f M. compare This will compare the assembly files in result against a reference set in result.ref. Any differences will be output to the screen. 5.5 Additional Operational Considerations None. 5.6 Sample Run 7 This is a sample run of gcc on a Sun machine: hostname% make -f M.sun make "MACHID=sun" "EXTRA_CFLAGS=-Bstatic" "OPT=-O2" validate rm -rf result mkdir result cp gcc result for i in jump.i varasm.i toplev.i gcc.i dbxout.i genrecog.i emit-rtl.i \ regclass.i recog.i genoutput.i explow.i print-tree.i stmt.i cexp.i tree.i \ insn-recog.i integrate.i optabs.i insn-emit.i; do cp input.ref/$i result; \ done Ignore syntax error from the following line for i in ; do ln $i result/$i; done sh: syntax error at line 1: `;' unexpected *** Error code 2 (ignored) cd result; /bin/time gcc -O -S -B../ jump.i varasm.i toplev.i gcc.i \ dbxout.i genrecog.i emit-rtl.i regclass.i recog.i genoutput.i explow.i \ print-tree.i stmt.i cexp.i tree.i insn-recog.i integrate.i optabs.i \ insn-emit.i > result.out 2> time.out cd result; for i in jump.s varasm.s toplev.s gcc.s dbxout.s genrecog.s \ emit-rtl.s regclass.s recog.s genoutput.s explow.s print-tree.s \ stmt.s cexp.s tree.s insn-recog.s integrate.s optabs.s insn-emit.s; \ do diff $i ../result.ref/$i; done hostname%