UnixBench Score: An Introduction

By Chao Qian (Xixie)

UnixBench is a benchmark suite that provides a basic indicators of the performance of a Unix-like system. It runs multiple tests on a system and compares the scores from a baseline system to produce an index value, which is combined to make an overall index for the system.

This article introduces UnixBench, focusing on how it calculates scores.

Running Parameters

Many customers directly run UnixBench after installation and use the last score for comparison.

The following is the sample result of a 4 core 8 GB CPU:

------------------------------------------------------------------------
Benchmark run: Mon, June 25, 2018 20:25:47 - 20:54:19
4 CPUs in system; running 1 parallel copy of tests
Dhrystone 2 using register variables 30971628.9 lps (10.0 s, 7 samples)
Double-Precision Whetstone 3928.1 MWIPS (12.4 s, 7 samples)
Execl Throughput 3117.6 lps (30.0 s, 2 samples)
File Copy 1024 bufsize 2000 maxblocks 645027.2 KBps (30.0 s, 2 samples)
File Copy 256 bufsize 500 maxblocks 229505.4 KBps (30.0 s, 2 samples)
File Copy 4096 bufsize 8000 maxblocks 1288742.6 KBps (30.0 s, 2 samples)
Pipe Throughput 1635960.9 lps (10.0 s, 7 samples)
Pipe-based Context Switching 337333.8 lps (10.0 s, 7 samples)
Process Creation 8238.2 lps (30.0 s, 2 samples)
Shell Scripts (1 concurrent) 5817.0 lpm (60.0 s, 2 samples)
Shell Scripts (8 concurrent) 2183.9 lpm (60.0 s, 2 samples)
System Call Overhead 2465754.7 lps (10.0 s, 7 samples)
System Benchmarks Index Values BASELINE RESULT INDEX
Dhrystone 2 using register variables 116700.0 30971628.9 2654.0
Double-Precision Whetstone 55.0 3928.1 714.2
Execl Throughput 43.0 3117.6 725.0
File Copy 1024 bufsize 2000 maxblocks 3960.0 645027.2 1628.9
File Copy 256 bufsize 500 maxblocks 1655.0 229505.4 1386.7
File Copy 4096 bufsize 8000 maxblocks 5800.0 1288742.6 2222.0
Pipe Throughput 12440.0 1635960.9 1315.1
Pipe-based Context Switching 4000.0 337333.8 843.3
Process Creation 126.0 8238.2 653.8
Shell Scripts (1 concurrent) 42.4 5817.0 1371.9
Shell Scripts (8 concurrent) 6.0 2183.9 3639.9
System Call Overhead 15000.0 2465754.7 1643.8
========
System Benchmarks Index Score 1362.9
------------------------------------------------------------------------
Benchmark Run: Mon, June 25, 2018 20:54:19 - 21:22:54
4 CPUs in system; running 4 parallel copies of tests
Dhrystone 2 using register variables 114984418.6 lps (10.0 s, 7 samples)
Double-Precision Whetstone 16614.2 MWIPS (11.6 s, 7 samples)
Execl Throughput 13645.3 lps (30.0 s, 2 samples)
File Copy 1024 bufsize 2000 maxblocks 751698.4 KBps (30.0 s, 2 samples)
File Copy 256 bufsize 500 maxblocks 230211.7 KBps (30.0 s, 2 samples)
File Copy 4096 bufsize 8000 maxblocks 1964420.6 KBps (30.0 s, 2 samples)
Pipe Throughput 5999380.0 lps (10.0 s, 7 samples)
Pipe-based Context Switching 1095000.2 lps (10.0 s, 7 samples)
Process Creation 34454.9 lps (30.0 s, 2 samples)
Shell Scripts (1 concurrent) 18218.1 lpm (60.0 s, 2 samples)
Shell Scripts (8 concurrent) 2493.1 lpm (60.0 s, 2 samples)
System Call Overhead 5643267.3 lps (10.0 s, 7 samples)
System Benchmarks Index Values BASELINE RESULT INDEX
Dhrystone 2 using register variables 116700.0 114984418.6 9853.0
Double-Precision Whetstone 55.0 16614.2 3020.8
Execl Throughput 43.0 13645.3 3173.3
File Copy 1024 bufsize 2000 maxblocks 3960.0 751698.4 1898.2
File Copy 256 bufsize 500 maxblocks 1655.0 230211.7 1391.0
File Copy 4096 bufsize 8000 maxblocks 5800.0 1964420.6 3386.9
Pipe Throughput 12440.0 5999380.0 4822.7
Pipe-based Context Switching 4000.0 1095000.2 2737.5
Process Creation 126.0 34454.9 2734.5
Shell Scripts (1 concurrent) 42.4 18218.1 4296.7
Shell Scripts (8 concurrent) 6.0 2493.1 4155.1
System Call Overhead 15000.0 5643267.3 3762.2
========
System Benchmarks Index Score 3357.0

Many people use 3357 for comparison and pay little attention to the single-process score 1362.9, which is also significant. The multi-process result is based on the number of CPU cores, while the single-process result is based on only one process. Both results show the system performance.

Then, what has been executed with . /Run? By default, the Index module is executed, which contains the following applets:

"dhry2reg", "whetstone-double", "execl",
"fstime", "fsbuffer", "fsdisk", "pipe", "context1", "spawn", "shell1", "shell8","syscall"

They correspond to the above results.

The source code indicates that the following parameters are executed: /Run Module -i Number of iterations -c Number of concurrent processes -q/-v (output mode)

  • Module: If you want only the test result of an applet, you can specify the module and modify the test content, which is convenient for debugging. You can also specify the execution program.
  • Number of iterations: By default, the framework supports two iteration modes, namely short iterations (3 iterations by default) and long iterations (10 iterations by default). The input number of iterations indicates the number of iterations by default in long iterations. In short iterations, the number of iterations is (Input number of iterations + 1)/3. If the result is less than 1, 1 is used. After the applet is configured, select the number of iterations.
  • Number of concurrent processes: If you want the number of concurrent processes to be different from the number of system CPUs, set this parameter.
  • Output mode: Select the Silent or Details output mode.

Score Calculation

When running each applet, you need to specify the iteration mode, execution time, and concurrency. The same mode is applied to all the processes each time.

Score Calculation with a Single Process

The following is an example of score calculation with a single process:

Process Creation -- 1 copy
==> "/opt/unixbench/UnixBench/pgms/spawn" 30 2>&1 >> "/opt/unixbench/UnixBench/results/VM_0_13_centos-2018-06-25-05.log"
#### Pass 1
# COUNT0: 247371 #Score
# COUNT1: 1 #timebase constant
# COUNT2: lps #Test case name
# elapsed: 30.003119 #Time consumed
# pid: 16803 #Process ID
# status: 0 #Whether exiting is successful
#### Pass 2
# COUNT0: 242919
# COUNT1: 1
# COUNT2: lps
# elapsed: 30.002898
# pid: 5035
# status: 0
#### Pass 3
# COUNT0: 243989
# COUNT1: 1
# COUNT2: lps
# elapsed: 30.002732
# pid: 21228
# status: 0
*Dump score: 242919.0
Count score: 243989.0
Count score: 247371.0
>>>> Results of 1 copy
>>>> score: 8188.34084738901
>>>> time: 30.0029255
>>>> iterations: 2

The following shows the method to obtain results of COUNT0, COUNT1, and COUNT2.

COUNT|x|y|x

After parsing, it is converted as shown in the following figure.

COUNT0 = x
COUNT1 = y
COUNT2 = z

The score calculation process is as follows:

1. The results are sorted in ascending order from COUNT0 and the last 1/3 of the results are removed.
2. If timebase is used, the time elapsed is always greater than 0, and $product += log($COUNT0) - log(Time elapsed/ $timebase). Otherwise, the time elapsed can be ignored and $product += log($COUNT0).
3. Single performance score: $score = exp($product/2), where 2 indicates the number of iterations. The calculated score is the same as the system-generated score.

The general calculation method is to first use the log to implement dimensionality reduction to narrow the gap between the results obtained through different numbers of iterations as much as possible. Then the factorial operation is applied to the average. This method also applies to multiple performance scores.

Multi-Process Score Calculation

If multiple concurrent processes are used for score calculation, see the following example:

Shell Scripts (1 concurrent) -- 4 copies
==> "/opt/unixbench/UnixBench/pgms/looper" 60 "/opt/unixbench/UnixBench/pgms/multi.sh" 1 2>&1 >> "/opt/unixbench/UnixBench/results/VM_0_13_centos-2018-06-25-05.log"
#### Pass 1
# COUNT0: 4614
# COUNT1: 60
# COUNT2: lpm
# elapsed: 60.005639
# pid: 20858
# status: 0
# COUNT0: 4596
# COUNT1: 60
# COUNT2: lpm
# elapsed: 60.009496
# pid: 20859
# status: 0
# COUNT0: 4592
# COUNT1: 60
# COUNT2: lpm
# elapsed: 60.011761
# pid: 20862
# status: 0
# COUNT0: 4614
# COUNT1: 60
# COUNT2: lpm
# elapsed: 60.011930
# pid: 20864
# status: 0
#### Pass 2
# COUNT0: 4547
# COUNT1: 60
# COUNT2: lpm
# elapsed: 60.005597
# pid: 10791
# status: 0
# COUNT0: 4590
# COUNT1: 60
# COUNT2: lpm
# elapsed: 60.013270
# pid: 10793
# status: 0
# COUNT0: 4578
# COUNT1: 60
# COUNT2: lpm
# elapsed: 60.006054
# pid: 10794
# status: 0
# COUNT0: 4561
# COUNT1: 60
# COUNT2: lpm
# elapsed: 60.014214
# pid: 10797
# status: 0
#### Pass 3
# COUNT0: 4631
# COUNT1: 60
# COUNT2: lpm
# elapsed: 60.013816
# pid: 31734
# status: 0
# COUNT0: 4632
# COUNT1: 60
# COUNT2: lpm
# elapsed: 60.012614
# pid: 31735
# status: 0
# COUNT0: 4637
# COUNT1: 60
# COUNT2: lpm
# elapsed: 60.005633
# pid: 31737
# status: 0
# COUNT0: 4645
# COUNT1: 60
# COUNT2: lpm
# elapsed: 60.006082
# pid: 31740
# status: 0
*Dump score: 18276.0
Count score: 18416.0
Count score: 18545.0
>>>> Sum of 4 copies
>>>> score: 18477.4244713467
>>>> time: 60.009621375
>>>> iterations: 2

The score calculation process is as follows:

  1. Score: For example, the score 18545 is the sum of the scores (4631 + 4632 + 4637 + 4645) of Pass1, Pass2, and Pass3.
  2. Time elapsed: This is an average, that is, (60.013816+60.012614+60.005633+60.006082)/4=60.00953625.
  3. timebase: COUNT1.

The preceding three steps combine concurrent calculation results into one single result. The subsequent calculation is the same as that for score calculation with a single process.

Total Score

Take the score calculation with multiple processes illustrated at the beginning of this article as an example.

  1. Each calculated score is a weighted score and must be divided by 10. (For single item competition, it doesn’t matter if the scores are divided by 10. This may aim to make the scores closer to the total score.)
  2. Calculation of the total score: =exp(average(Each score*10)). The expected result is obtained.

Summary

This is the end of the description of score calculation with UnixBench. The tables below illustrate the calculation process of the scores.

Single Process Score

Multi Process Score

Total Score

Reference:https://www.alibabacloud.com/blog/unixbench-score-an-introduction_594677?spm=a2c41.12761715.0.0

Written by

Follow me to keep abreast with the latest technology news, industry insights, and developer trends.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store