Does and Doesn't Do
Nearly all benchmarks are designed for a specific purpose. Quite often,
however, users broaden that purpose beyond what the benchmark is designed
to do, or in some cases they assume the benchmark can't do something that
it actually can do. The
benchmark is no different: it has been both overextended and under-appreciated,
sometimes reducing its overall value to the user. Here is a closer look
at SPECviewperf: what it is and what it can and can't do.
SPECviewperf measures the 3D rendering performance of systems running
under OpenGL. The SPECopcSM project group
has worked with independent software vendors (ISVs) to obtain tests, data
sets and weights that constitute what is called a viewset. Each viewset
represents the graphics rendering portion of an actual application. The
ISVs that develop SPECopc viewsets have provided percentage weights for
each test for which a performance number is reported. ISVs have defined
these percentages to indicate the relative importance of a test within
the overall application.
SPECviewperf offers the following characteristics:
Several factors make SPECviewperf unique from other benchmarks:
It provides a single-source code for apples-to-apples comparison and performance
tuning across different hardware platforms.
It runs on multiple operating systems, including Windows 95, OS/2, UNIX
and Windows NT.
It runs across different processors, including Alpha, Intel, MIPS and PowerPC.
It runs on multiple windowing environments, including Presentation Manager,
X and Windows.
It encompasses a wide variety of OpenGL features and rendering techniques.
It is easily accessible through the SPECopc project subcommittee, ftp and
through OpenGL sample disk distribution.
It uses datasets that are designed for and used by real applications.
It uses rendering parameters and models selected by independent software
vendors (ISVs) and graphics users.
It produces numbers based on frames per second, a measurement with which
users can readily identify.
It provides one number for each rendering path using one data set.
What SPECviewperf Measures
SPECviewperf measures performance for the following entities:
3D primitives, including points, lines, line_strip, line_loop, triangles,
triangle_strip, triangle_fan, quads and polygons;
attributes per vertex, per primitive and per frame;
The Five-Step Program
SPECviewperf is not a single-number benchmark. In order to use it to its
fullest advantage, ISVs and users need to relate the benchmark to their
actual applications. Here are the five steps recommended for using SPECviewperf
Identify software code paths that are important to the application.
Identify the primitives used within the application.
Select datasets that are most appropriate to the application. The datasets
should reflect the level of geometry and rasterization found in the application.
Identify attributes and the level at which they are applied (per vertex,
per primitive or per frame).
Assign a weight to each path based on the percentage of time in each path
and the importance of the path to the application.
What SPECviewperf Can't Do
Although SPECviewperf is a good tool for measuring OpenGL performance as
it relates to applications, like all benchmarks it has limitations. Most
important of these is that it cannot be used to compare performance across
different application programming interfaces (APIs). Also, it does not
run itself; users must participate in the benchmarking process. When testing
and reporting results, SPECviewperf does not account for the following
effects caused by switching primitives,
input effects on the event loop,
user interface rendering and management,
complex motion of multiple models,
effects of CPU load on the graphics subsystem,
color index visual performance, and
multi-context, multi-window effects.
Development of SPECviewperf within the SPECopc group is an ongoing process,
with future enhancements designed to address key graphics applications
issues not covered by the current benchmark.