What is This Thing Called "SPECviewperf™"?

SPECviewperf™ is a portable OpenGL performance benchmark program written in C. It was developed by IBM. Later updates and significant contributions were made by SGI, Digital (Compaq, HP), 3Dlabs (Creative Labs) and other SPECopcSM project group members. SPECviewperf provides a vast amount of flexibility in benchmarking OpenGL performance. Currently, the program runs on most implementations of UNIX, Windows XP, Windows 2000, and Linux.

The OpenGL Performance Characterization (SPECopc) project group has endorsed SPECviewperf as its first OpenGL benchmark. Performance numbers based on SPECviewperf were first published in the Q4 1994 issue of The GPC Quarterly.

SPECopc group member companies have ported the SPECviewperf code to their operating systems and window environments. The SPECopc project group maintains a single source code version of the SPECviewperf code that is available to the public.

Benchmarking with SPECviewperf

SPECviewperf parses command lines and data files, sets the rendering state, and converts data sets to a format that can be traversed using OpenGL rendering calls. It renders the data set for a pre-specified amount of time or number of frames with animation between frames. Finally, it outputs the results.

SPECviewperf reports performance in frames per second. Other information about the system under test -- all the rendering states, the time to build display lists (if applicable), and the data set used -- are also output in a standardized report.

A "benchmark" using SPECviewperf is really a single invocation of SPECviewperf with command-line options telling the SPECviewperf program which data set to read in, which texture file to use, what OpenGL primitive to use to render the data set, which attributes to apply and how frequently, whether or not to use display lists, and so on. One quickly realizes that there are an infinite number of SPECviewperf "benchmarks" (an infinite number of data sets multiplied by an almost infinite number of command-line states).

Real-World Benchmarking

SPECopc project group members recognize the importance of real-world benchmarks. >From the beginning, the group has sought benchmarks representative of the OpenGL rendering portion of independent software vendor (ISV) applications. Along these lines, the project group has come up with what it calls a viewset. A viewset is a group of individual runs of SPECviewperf that attempt to characterize the graphics rendering portion of an ISV's application.

Viewsets are generally not developed by the SPECopc project group; they come from the ISVs themselves. Members of the SPECopc project group often "sponsor" the ISV. Sponsorship entails helping the ISV in several areas, including how to obtain the SPECviewperf code, how to convert data sets to a SPECviewperf format, how to use SPECviewperf, how to create SPECviewperf tests to characterize the application, how to determine weights for each of the individual SPECviewperf tests based on application usage, and finally to help offer the viewset to the SPECopc project group for consideration as a standard SPECopc viewset. Any ISV wishing to develop a viewset should contact an SPECopc representative listed on the GPC Organization page.

Currently, there are six standard SPECopc application viewsets:

These type of applications typically render large data sets. All six viewsets represent relatively high-end applications.

The input to SPECviewperf is the command line. Here are the current command line options for SPECviewperf 7.0:

Program options:

Viewperf v7.0

Program options:
-polygon -pg   <file>  : Viewpoint object to be used in the tests
-triangle -tr  <file>  : Viewpoint object to be used in the tests
-quad -qd      <file>  : Viewpoint object to be used in the tests
-mesh -mh      <file>  : Mesh object to be used in the tests
-frameattr -fa <file>  : Mesh3 attributes for each frame
-rendermode -rm <mode> : POINT, VECTOR, LINE, POLYGON, TMESH, TFAN, TRIANGLE, or QUAD - default LINE
-vcriteria -vcrit      : AUX Visual selection criteria - EXACT, MIN
- default MIN
-vid <id>              : Ask AUX for visual with ID = <id>
-vaccum -vac           : Ask AUX for an accumulation buffer visual
-valpha -val           : Ask AUX for an alpha buffer visual
-vdepthbuffer -vz      : Ask AUX for a depth buffer visual
-vstencil -vst         : Ask AUX for a stencil buffer visual
-indirectrender -ir    : Render indirect - default direct
-nodither -ndi         : Disable dithering
-ortho -or             : Parallel/Orthographic projection - default Perspective
-displaylist -dl       : Render with display list mode
-vertexarray -va       : Render with vertex array mode (OpenGL1.1)
-interleaved -il       : Packing vertex data interleaved in memory
-colorper -cp  <mode>  : FRAME = Color per Frame, PRIMITIVE = Color per Primitive, VERTEX = Color per Vertex - default FRAME
-texture -tx   <file>  : Image for texturing
-texgen -txg <file> <mode> : <file> is image for enviroment mapping <mode> is SPHERE_MAP, OBJECT_LINEAR, EYE_LINEAR - default EYE_LINEAR
-texobj -txo <file>    : Texture object mode <file> is list of images
-magfilter -magf <flt> : NEAREST, LINEAR - default NEAREST
-texenv -te <env>      : Texture enviroment, MODULATE, DECAL, BLEND - default DECAL
-texcomp -tc <num>     : Texture components where <num> is 1,2,3, or 4 -default 3
-blend -bl             : Enable Blending
-linewidth -lw <width> : Linewidth for wire/vector tests - default 1.0
-xwinsize -xws <side>  : Size of test windows X dimension - default 700
-ywinsize -yws <side>  : Size of test windows Y dimension - default 700
-numframes -nf <num>   : Number of frames to be rendered during measurement Takes priority over -mp
-numilights -nil <num> : Turns on <num> infinite lights - default 0
-numllights -nll <num> : Turns on <num> local lights - default 0
-colormaterial -cm <side> <mode>: <side> is FRONT, BACK, FRONT_AND_BACK - default FRONT <mode> is AMBIENT, DIFFUSE, EMISSION, SPECULAR, AMBIENT_AND_DIFFUSE - default AMBIENT_AND_DIFFUSE
-backface -bf          : Cull Backfacing primitives - default off
-frontface -ff         : Cull Frontfacing primitives - default off
-singlebuffer -sb      : Single buffer mode
-fog -fg               : Enable fog
-linesmooth -ls        : Enable line antialiasing
-polysmooth -ps        : Enable polygon antialiasing
-facetnormal -fn       : Use facet normals when lighting
-linestipple -lp       : Enable line stipple
-polystipple -pp       : Enable polygon stipple
-batch -bt <num>       : Batch <num> primitives together per glBegin/glEnd Valid with POINT, VECTOR, TRIANGLE, and QUADS
-polymodefront -pmf    : POINT, LINE, or FILL - default FILL
-polymodeback  -pmb    : POINT, LINE, or FILL - default FILL
-flat -f               : Set shademodel to FLAT - default GOURAUD
-zbuffer -zb           : Enable zbuffer for tests - default off
-clip -c               : Align object on 3D clip boundary
-lighttwoside -l2s     : Light both sides of model
-localview -lv         : Define local viewer for lit tests
-normalize -nm         : Enable normal normalization
-minperiod -mp <num>   : Set minimum testing period in seconds
-mblur <num>           : Use motion blur with num being amount of decay
-aa_multi <x> <r>      : Full scene antialiasing rendered x times at an offset of r.  r should be tuned to the viewset
-walkthru -wt          : Walkthru mode
-geomrepeat -gr        : Geometry repeat
-grab <script>	       : Screen capture
-grabonly -go          : Don't run benchmark, just do screen capture
-swap <true/false>     : Force byte swapping
-notitle -nt           : Disable title screen
-startframe -sf <num>  : Set starting frame of a walkthru
-framepause -fp        : Pause after each frame waiting for enter key
-newview -nv           : Enable new view frustum calculation
-fov <angle>           : Set field of view in degrees
-spin x y z dx dy dz   : Set rotation initial values and speeds (degrees)
-clipplane             : Enables a user clip plane
-nomaterial            : Disables material changes if present
-nolight               : Disables viewperf lighting
-forcecalibrate        : Forces viewperf calibration
-tracelights -tl       : Use lights generated in scene
-norejection           : Turns off trivial rejection for walk through.
-noframecounter        : Turn off display of frame counter 
The -threads option is applicable only if SPECviewperf has been compiled with the C Preprocessor variable MP defined. SPECviewperf should only be compiled this way if the vendor supplied makefile has defined the MP flag (-DMP).

Here is an example of the command line from test #3 of the former CDRS-03 viewset:
viewperf -mh mower-ts -dl -rm TMESH -vz -bl -zb -nil 1 -cp PRIMITIVE -mp 10 -nf 10

This command line is providing the following directions:

The following significant defaults are in effect during this test: direct rendering, perspective viewing, no texture mapping, window size is 700 x 700 pixels, no culling, double buffering, smooth shading, no clipping, and single-sided lighting.

Upon entering the command line, the system under test will run the benchmark for at least 10 seconds or 10 frames. This is to insure a valid test result. During the test, the SPECviewperf code renders the data set, mower-ts.msh, which is half of the shell of a lawnmower, rotating it about the center of the window with the above specified characteristics.

What follows is the output of SPECviewperf 6.1 running the first test from ProCDRS-01 on an SGI OCTANE/MXE:

    Viewperf Version                                   6.1
    Viewperf Arguments                                 -mh newtruck-wire -dl -linesmooth -srcblendfunc SRC_ALPHA -dstblendfunc ONE_MINUS_SRC_ALPHA -f -xws 1260 -yws 980 -newview -mp 20 -grab grab1.scr 
    Month                                              11
    Day                                                6
    Year                                               1998
    Host                                               pptemp5
    Operating System                                   IRIX64
    Operating System Version                           6.5 07271714
    Host Vendor                                        SGI
    Host Model                                         IP30
    Host CPU                                           250 MHz R10000
    Host CPU Count                                     2
    Host Memory Size (MB)                              128
    Host Primary Cache Size (KB)                       32/32 (D/I)
    Host Secondary Cache Size (KB)                     1024 (D+I)
    Window System                                      X11R5
    Driver Version                                     N/A
    OpenGL Vendor                                      SGI
    OpenGL Version                                     1.1 Irix 6.5
    OpenGL Extensions                                  GL_EXT_abgr GL_EXT_blend_color GL_EXT_blend_logic_op GL_EXT_blend_minmax GL_EXT_blend_subtract GL_EXT_convolution GL_EXT_copy_texture GL_EXT_histogram GL_EXT_packed_pixels GL_EXT_polygon_offset GL_EXT_subtexture GL_EXT_texture GL_EXT_texture3D GL_EXT_texture_object GL_EXT_vertex_array GL_SGI_color_matrix GL_SGI_color_table GL_SGI_texture_color_table GL_SGIS_texture_filter4 GL_SGIS_detail_texture GL_SGIS_texture_border_clamp GL_SGIS_texture_select GL_SGIS_texture_lod GL_SGIX_pixel_texture GL_SGIX_texture_multi_buffer 
    OpenGL Renderer                                    IMPACT/2/2/4
    OpenGL Client Vendor                               SGI
    OpenGL Client Version                              1.2 Irix 6.5
    OpenGL Client Extensions                           GLX_EXT_import_context GLX_EXT_visual_info GLX_EXT_visual_rating GLX_SGI_make_current_read GLX_SGI_swap_control GLX_SGI_video_sync GLX_SGIS_multisample GLX_SGIX_fbconfig GLX_SGIX_pbuffer GLX_SGIX_swap_barrier GLX_SGIX_swap_group GLX_SGIX_video_resize GLX_SGIX_video_source 
    GLU Version                                        1.2 Irix 6.5
    GLU Extensions                                     GLU_EXT_abgr GLU_EXT_nurbs_tessellator GLU_EXT_object_space_tess GLU_EXT_packed_pixels GLU_EXT_texture GLU_SGI_filter4_parameters 
    Direct Rendering                                   True
    Double Buffer                                      True
    Stereo                                             False
    RGBA                                               True
    Color Index Size                                   -1
    Red Size                                           12
    Green Size                                         12
    Blue Size                                          12
    Alpha Size                                         0
    Accum Red Size                                     16
    Accum Green Size                                   16
    Accum Blue Size                                    16
    Accum Alpha Size                                   16
    Depth Size                                         0
    Stencil Size                                       0
    Auxiliary Buffer Count                             0
    Frame BufferLevel                                  0
    Visual ID                                          0X38
    Visual Class                                       TrueColor
    Window Width (pixels)                              1260
    Window Height (pixels)                             980
    Screen Width (pixels)                              1280
    Screen Height (pixels)                             1024
    Display                                            :0.0
    OpenGL Server Vendor                               SGI
    OpenGL Server Version                              1.2 Irix 6.5
    OpenGL Server Extensions                           1.2 Irix 6.5
    GLX Server Version                                 1.1
    GLX Server Extensions                              GLX_EXT_import_context GLX_EXT_visual_info GLX_EXT_visual_rating GLX_SGI_make_current_read GLX_SGI_swap_control GLX_SGI_video_sync GLX_SGIX_fbconfig GLX_SGIX_pbuffer GLX_SGIX_swap_group 
    Screen Number                                      0
    Shared Memory Connection                           False
    Visual Selection Criteria                          MINIMUM
    Number of Execution Threads                        1
    Geometry File                                      newtruck-wire
    Input Mode                                         -mh 
    Minimum Test Period                                20.000000
    Number of Frames                                   0
    Number of Primitives                               18691
    Number of Vertices per Frame                       202481
    Number of Vertices per Primitive                   10.833075
    Toggle Mode                                        NONE
    Batching Count                                     0
    Render Mode                                        MIXED
    Color per                                          COLOR_PER_FRAME
    Orthographic Projection                            FALSE
    Display List                                       TRUE
    Vertex Array (OpenGL1.1)                           FALSE
    Interleaved Data                                   FALSE
    Clip Geometry                                      FALSE
    Walkthrough Mode                                   FALSE
    Back Face Cull                                     FALSE
    Front Face Cull                                    FALSE
    Front Polygon Mode                                 FILL
    Back Polygon Mode                                  FILL
    Polygon Stipple Enable                             FALSE
    Polygon Antialiasing Enable                        FALSE
    Line Width                                         1.000000
    Line Stipple Enable                                FALSE
    Line Antialiasing Enable                           TRUE
    Number of Infinite Lights                          0
    Number of Local Lights                             0
    Color Material Enable                              FALSE
    Color Material Face                                FRONT
    Color Material Mode                                AMBIENT_AND_DIFFUSE
    Facet Normals                                      FALSE
    Two Sided Lighting Enable                          FALSE
    Local Viewer Enable                                FALSE
    Flat Shading                                       TRUE
    Fog Enable                                         FALSE
    Normal Normalization                               FALSE
    Texture Enable                                     FALSE
    TexObj Enable                                      FALSE
    Texture Generation Mode                            NO_TEXTURE_GENERATION
    Texture File                                       NONE
    Texture Minification Filter                        NEAREST
    Texture Magnification Filter                       NEAREST
    Texture Environment Mode                           DECAL
    Texture Components                                 3
    Depth Test Enable                                  FALSE
    Blend Enable                                       FALSE
    Source Blend Function                              SRC_ALPHA
    Destination Blend Function                         ONE_MINUS_SRC_ALPHA
    Dithering Enable                                   TRUE
    Motion Blur Amount                                 0.000000
    Full Scene Antialiasing Redraws                    0
    Full Scene Antialiasing Jitter Amount              0.000000
  0.0900        sec (DL Build)  --      -mh newtruck-wire -rm MIXED -mp 20 -cp FRAME -dl -f -ls -xws 1260 -yws 980 -nv 
                Number of frames run: 406, Test period: 20.120001 (sec)
    20.2        frames/sec      --      -mh newtruck-wire -rm MIXED -mp 20 -cp FRAME -dl -f -ls -xws 1260 -yws 980 -nv 

The output provides all the SPECviewperf state information, as well as the configuration parameters for the system under test. The display list build time was 0.09 seconds, the number of frames was 406, and the actual test time was 20.12 seconds.

SPECviewperf does a test to see how many frames need to be rendered to meet the minimum time period. In this case, the calibration calculates the number of frames necessary to meet the minimum, bumps up the number of frames if the minimum is not met, then reruns the test.

The last line is the number of frames per second, which is the fundamental metric for each test. This is the result that is printed in the summary and individual report pages within the GPC News SPECopc section. Also printed in the summary and individual report pages is a single composite number for each viewset. This composite number is derived using a weighted geometric mean methodology (essentially the product of the results of each test raised to the power of the test weightings).

SPECviewperf 7.0

The latest version of SPECviewperf includes a number of new features intended to allow it to more closely model the performance of OpenGL graphics applications. The new features include support for generate state changes made during frames and data traversal, facet color processing, and display of lighting and textures from frame to frame. For more information, please read the documentation provided with the SPECviewperf source code.

Rob Putney / Simon Green / Chris Watts / Bill Licea-Kane / Allen Jensen