IBM POWER9 Preliminary Benchmarks after CVE-2020–4788 mitigations

Much is being talked about the latest vulnerability mitigations on L1 cache for IBM POWER processors addressed in CVE-2020–4788.

Update on 23/11: This post has been updated to address Pgbench tests that provides more realistic numbers related to the CVE and mitigations. Check the PGbench section below.

The intent of this post is to run some preliminary benchmarks with and without such mitigations. There is no scientific method for the tests other than full reboots and replication of the scenarios between versions. For complete tests, check out in the future for Phoronix benchmarks.

The tests were performed on a Raptor Computing Systems Blackbird with a 4-core, 16-thread POWER9 CPU (02CY650) and 8GB RAM.

Description of the patch and vulnerability: https://patchwork.ozlabs.org/project/linuxppc-dev/cover/20201119231333.361771-1-dja@axtens.net/

The Linux commit that merged the mitigation is: dda3f4252e6c N 2 days ago Linus..Merge tag 'powerpc-cve-2020-4788' of git://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux.

Patch series that added the mitigation: https://patchwork.ozlabs.org/project/linuxppc-dev/list/?series=215657

Current mitigation list for the kernel can be found here.

Tests were performed with:

  • Kernel 5.9.9 from stable tree with mitigations=off kernel parameter.
  • Kernel 5.10-rc4 (commit 27bba9c5) from Linus tree with mitigations=off kernel parameter.
  • Kernel 5.10-rc4 (commit 27bba9c5) from Linus tree without the mitigations=off kernel parameter.

TLDR

Here’s a quick resume of the numbers I found. As can be seen, on CPU heavy workloads with little context switching, no significant difference has been found with the mitigations and between the tested Linux Kernel versions.

Bigger value is better.

With heavy contest-switching applications such as PostgreSQL database with multiple clients, I saw a performance hit for 4–5% or even less with the new Kernel:

Of course detailed benchmarks are advised with your workloads and real-world scenarios but in my opinion, these tests shows that the performance impact are not as big as initially expected.

Pgbench Benchmarks

Adding to previous tests below and by a suggestion from Karel Gardas, I've benchmarked the Kernel differences using Pgbench which is a bundled application from PostgreSQL to benchmark the database.

I've compared Kernel 5.9.9 (that didn't have the mitigations) with 5.9.10 that already had the mitigations backported and enabled, compared 5.9.10 with and without the mitigations enabled and then compared 5.10-rc5 with and without the mitigations enabled.

The results show the amount of TPS (transactions per seconds) the pgbench application achieved on the database.

I've tested two scenarios, one with PostgreSQL running on a Docker containers with 2 CPUs and the pgbench using 2 threads and another set of tests with the container with 6 CPUs and pgbench with 4 threads. Each pgbench run simulated 60 clients and 2000 transactions each.

As can be seen, the standard deviation in values is bigger with more resources allocated due to variations because both the database and the pgbench application are running in the same server.

PostgreSQL 2 CPUs — Pgbench 2 threads

# Start PostgreSQL container with 2 CPUs
docker run --name postgres --cpus 2 -e POSTGRES_PASSWORD=mypass -d postgres
# Enter it's shell
docker exec -it -u postgres postgres bash
# Create test database
psql
CREATE DATABASE mydb;
exit
# Populate the database
pgbench -U postgres -i -s 50 mydb
# Run 10 test iterations with 60 clients and 2000 transactions each
for x in 1 2 3 4 5 6 7 8 9 10; do
echo "-------------- Run $x -------------- ";
pgbench -U postgres -c 60 -j 2 -t 2000 mydb;
done

As seen in this test, the standard deviation is pretty low for all cases. Also we see that in my tests, the mitigations impose about 4–5% performance impact on Kernel 5.9.10 and 1.3% on Kernel 5.10-rc. Also compared Kernel 5.9.9 where mitigations were not implemented to 5.9.10 with mitigations enabled imposing about 2.8% performance hit.

PostgreSQL 6CPUs — Pgbench 4threads

# Start PostgreSQL container with 2 CPUs
docker run --name postgres --cpus 6-e POSTGRES_PASSWORD=mypass -d postgres
# Enter it's shell
docker exec -it -u postgres postgres bash
# Create test database
psql
CREATE DATABASE mydb;
exit
# Populate the database
pgbench -U postgres -i -s 50 mydb
# Run 10 test iterations with 60 clients and 2000 transactions each
for x in 1 2 3 4 5 6 7 8 9 10; do
echo "-------------- Run $x -------------- ";
pgbench -U postgres -c 60 -j 4 -t 2000 mydb;
done

As seen in this second set, the standard deviation is way higher for all cases meaning the benchmark imposes a performance hit in the running database. Also we see that the numbers are more sparse here and the mitigations impose about 1.6% performance impact on Kernel 5.9.10 and less than 1% on Kernel 5.10-rc. Also compared Kernel 5.9.9 where mitigations were not implemented to we see no practical performance hit.

7Zip and SPECjvm2008 Benchmarks

Kernel 5.9.9 with mitigations=off

Built from https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git, tag v5.9.9.

Linux debian-blackbird 5.9.9 #2 SMP Fri Nov 20 20:02:18 -03 2020 ppc64le GNU/Linux❯ cat /proc/cmdline
root=UUID=cc48bfea-adf6-4825-9908-d28de9b9d420 ro mitigations=off

Vulnerabilities:

grep -H '' /sys/devices/system/cpu/vulnerabilities/*
/sys/devices/system/cpu/vulnerabilities/itlb_multihit:Not affected
/sys/devices/system/cpu/vulnerabilities/l1tf:Vulnerable: L1D private per thread
/sys/devices/system/cpu/vulnerabilities/mds:Not affected
/sys/devices/system/cpu/vulnerabilities/meltdown:Vulnerable: L1D private per thread
/sys/devices/system/cpu/vulnerabilities/spec_store_bypass:Vulnerable
/sys/devices/system/cpu/vulnerabilities/spectre_v1:Vulnerable, ori31 speculation barrier enabled
/sys/devices/system/cpu/vulnerabilities/spectre_v2:Mitigation: Indirect branch serialisation (kernel only)
/sys/devices/system/cpu/vulnerabilities/srbds:Not affected
/sys/devices/system/cpu/vulnerabilities/tsx_async_abort:Not affected

7Zip benchmark

❯ 7z b7-Zip 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21
p7zip Version 16.02 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,64 bits,16 CPUs LE)
LE
CPU Freq: 1750 1878 1871 1861 1878 1878 1875 1877 1878
RAM size: 7664 MB, # CPU hardware threads: 16
RAM usage: 3530 MB, # Benchmark threads: 16
Compressing | Decompressing
Dict Speed Usage R/U Rating | Speed Usage R/U Rating
KiB/s % MIPS MIPS | KiB/s % MIPS MIPS
22: 22193 1170 1846 21590 | 303516 1591 1627 25887
23: 22350 1228 1854 22772 | 298372 1587 1627 25816
24: 22278 1272 1883 23954 | 292911 1594 1613 25710
| 25: 22002 1302 1929 25122 | 287272 1588 1610 25566 |
| ---------------------------------- | -------------------------- |
| Avr: 1243 1878 23359 | 1590 1619 25745 |
Tot: 1417 1748 24552

Java SPECjvm2008 running on OpenJDK 8u181 in a Docker container

Docker command: docker run -it --rm -v $(pwd):/test openjdk:8u181-jdk-stretch bash

Specjvm2008 compress benchmark:

java -jar SPECjvm2008.jar -wt 30s -it 1m -bt 16 -i 3 -ikv -ict compressSPECjvm2008 Peak
Properties file: none
Benchmarks: compress
WARNING: Run will not be compliant.
Property specjvm.run.initial.check must be true for publication.
Property specjvm.maxiter must be 1 for publication.
Property specjvm.miniter must be 1 for publication.
Not a compliant sequence of benchmarks for publication.
Property specjvm.run.checksum.validation must be true for publication.
Property specjvm.iteration.time must be at least 240 seconds for publication.
--- --- --- --- --- --- --- --- --- Benchmark: compress
Run mode: timed run
Test type: multi
Threads: 16
Warmup: 30s
Iterations: 3
Run length: 60s
Warmup (30s) begins: Sat Nov 21 20:55:24 UTC 2020
Warmup (30s) ends: Sat Nov 21 20:55:59 UTC 2020
Warmup (30s) result: 294.42 ops/m
Iteration 1 (60s) begins: Sat Nov 21 20:55:59 UTC 2020
Iteration 1 (60s) ends: Sat Nov 21 20:57:03 UTC 2020
Iteration 1 (60s) result: 300.92 ops/m
Iteration 2 (60s) begins: Sat Nov 21 20:57:03 UTC 2020
Iteration 2 (60s) ends: Sat Nov 21 20:58:07 UTC 2020
Iteration 2 (60s) result: 301.22 ops/m
Iteration 3 (60s) begins: Sat Nov 21 20:58:07 UTC 2020
Iteration 3 (60s) ends: Sat Nov 21 20:59:12 UTC 2020
Iteration 3 (60s) result: 297.42 ops/m
Valid run!
Score on compress: 301.22 ops/m
Results are stored in:
/test/java/SPECjvm2008/results/SPECjvm2008.058/SPECjvm2008.058.raw
Generating reports in:
/test/java/SPECjvm2008/results/SPECjvm2008.058
Noncompliant composite result: 301.22 ops/m

Specjvm2008 mpegaudio benchmark

java -jar SPECjvm2008.jar -wt 30s -it 1m -bt 16 -i 3 -ikv -ict mpegaudioSPECjvm2008 Peak
Properties file: none
Benchmarks: mpegaudio
WARNING: Run will not be compliant.
Property specjvm.run.initial.check must be true for publication.
Property specjvm.maxiter must be 1 for publication.
Property specjvm.miniter must be 1 for publication.
Not a compliant sequence of benchmarks for publication.
Property specjvm.run.checksum.validation must be true for publication.
Property specjvm.iteration.time must be at least 240 seconds for publication.
--- --- --- --- --- --- --- --- --- Benchmark: mpegaudio
Run mode: timed run
Test type: multi
Threads: 16
Warmup: 30s
Iterations: 3
Run length: 60s
Warmup (30s) begins: Sat Nov 21 21:01:12 UTC 2020
Warmup (30s) ends: Sat Nov 21 21:01:52 UTC 2020
Warmup (30s) result: 162.51 ops/m
Iteration 1 (60s) begins: Sat Nov 21 21:01:52 UTC 2020
Iteration 1 (60s) ends: Sat Nov 21 21:02:58 UTC 2020
Iteration 1 (60s) result: 174.59 ops/m
Iteration 2 (60s) begins: Sat Nov 21 21:02:58 UTC 2020
Iteration 2 (60s) ends: Sat Nov 21 21:04:04 UTC 2020
Iteration 2 (60s) result: 174.62 ops/m
Iteration 3 (60s) begins: Sat Nov 21 21:04:04 UTC 2020
Iteration 3 (60s) ends: Sat Nov 21 21:05:10 UTC 2020
Iteration 3 (60s) result: 174.55 ops/m
Valid run!
Score on mpegaudio: 174.62 ops/m
Results are stored in:
/test/java/SPECjvm2008/results/SPECjvm2008.059/SPECjvm2008.059.raw
Generating reports in:
/test/java/SPECjvm2008/results/SPECjvm2008.059
Noncompliant composite result: 174.62 ops/m

Specjvm2008 crypto.rsa benchmark

java -jar SPECjvm2008.jar -wt 30s -it 1m -bt 16 -i 3 -ikv -ict crypto.rsaSPECjvm2008 Peak
Properties file: none
Benchmarks: crypto.rsa
WARNING: Run will not be compliant.
Property specjvm.run.initial.check must be true for publication.
Property specjvm.maxiter must be 1 for publication.
Property specjvm.miniter must be 1 for publication.
Not a compliant sequence of benchmarks for publication.
Property specjvm.run.checksum.validation must be true for publication.
Property specjvm.iteration.time must be at least 240 seconds for publication.
--- --- --- --- --- --- --- --- --- Benchmark: crypto.rsa
Run mode: timed run
Test type: multi
Threads: 16
Warmup: 30s
Iterations: 3
Run length: 60s
Warmup (30s) begins: Sat Nov 21 21:09:46 UTC 2020
Warmup (30s) ends: Sat Nov 21 21:10:17 UTC 2020
Warmup (30s) result: 1795.27 ops/m
Iteration 1 (60s) begins: Sat Nov 21 21:10:17 UTC 2020
Iteration 1 (60s) ends: Sat Nov 21 21:11:18 UTC 2020
Iteration 1 (60s) result: 1816.65 ops/m
Iteration 2 (60s) begins: Sat Nov 21 21:11:18 UTC 2020
Iteration 2 (60s) ends: Sat Nov 21 21:12:19 UTC 2020
Iteration 2 (60s) result: 1818.78 ops/m
Iteration 3 (60s) begins: Sat Nov 21 21:12:19 UTC 2020
Iteration 3 (60s) ends: Sat Nov 21 21:13:19 UTC 2020
Iteration 3 (60s) result: 1818.64 ops/m
Valid run!
Score on crypto.rsa: 1818.78 ops/m
Results are stored in:
/test/java/SPECjvm2008/results/SPECjvm2008.060/SPECjvm2008.060.raw
Generating reports in:
/test/java/SPECjvm2008/results/SPECjvm2008.060
Noncompliant composite result: 1818.78 ops/m

Kernel 5.10.0-rc4 with mitigations (no mitigations=off in cmdline)

Built from https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git, commit 27bba9c5 (pre 5.10.0-rc5).

Linux debian-blackbird 5.10.0-rc4 #3 SMP Sat Nov 21 11:56:17 -03 2020 ppc64le GNU/Linux❯ cat /proc/cmdline
root=UUID=cc48bfea-adf6-4825-9908-d28de9b9d420 ro

Mitigations:

❯ fgrep -r '' /sys/devices/system/cpu/vulnerabilities
/sys/devices/system/cpu/vulnerabilities/spectre_v2:Mitigation: Indirect branch serialisation (kernel only)
/sys/devices/system/cpu/vulnerabilities/itlb_multihit:Not affected
/sys/devices/system/cpu/vulnerabilities/mds:Not affected
/sys/devices/system/cpu/vulnerabilities/l1tf:Mitigation: RFI Flush, L1D private per thread
/sys/devices/system/cpu/vulnerabilities/spec_store_bypass:Mitigation: Kernel entry/exit barrier (eieio)
/sys/devices/system/cpu/vulnerabilities/tsx_async_abort:Not affected
/sys/devices/system/cpu/vulnerabilities/spectre_v1:Mitigation: __user pointer sanitization, ori31 speculation barrier enabled
/sys/devices/system/cpu/vulnerabilities/srbds:Not affected
/sys/devices/system/cpu/vulnerabilities/meltdown:Mitigation: RFI Flush, L1D private per thread

7Zip benchmark

❯ 7z b7-Zip 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21
p7zip Version 16.02 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,64 bits,16 CPUs LE)
LE
CPU Freq: 1745 1878 1878 1879 1877 1879 1878 1878 1877
RAM size: 7656 MB, # CPU hardware threads: 16
RAM usage: 3530 MB, # Benchmark threads: 16
Compressing | Decompressing
Dict Speed Usage R/U Rating | Speed Usage R/U Rating
KiB/s % MIPS MIPS | KiB/s % MIPS MIPS
22: 21371 1126 1847 20790 | 302200 1586 1625 25775
23: 22437 1239 1845 22861 | 297196 1580 1627 25714
24: 22281 1278 1875 23957 | 292134 1591 1612 25642
| 25: 21968 1303 1926 25082 | 287146 1588 1610 25555 |
| ---------------------------------- | -------------------------- |
| Avr: 1236 1873 23173 | 1586 1619 25671 |
Tot: 1411 1746 24422

Java SPECjvm2008 running on OpenJDK 8u181 in a Docker container

Docker command: docker run -it --rm -v $(pwd):/test openjdk:8u181-jdk-stretch bash

Specjvm2008 compress benchmark

java -jar SPECjvm2008.jar -wt 30s -it 1m -bt 16 -i 3 -ikv -ict compressSPECjvm2008 Peak
Properties file: none
Benchmarks: compress
WARNING: Run will not be compliant.
Property specjvm.run.initial.check must be true for publication.
Property specjvm.maxiter must be 1 for publication.
Property specjvm.miniter must be 1 for publication.
Not a compliant sequence of benchmarks for publication.
Property specjvm.run.checksum.validation must be true for publication.
Property specjvm.iteration.time must be at least 240 seconds for publication.
--- --- --- --- --- --- --- --- --- Benchmark: compress
Run mode: timed run
Test type: multi
Threads: 16
Warmup: 30s
Iterations: 3
Run length: 60s
Warmup (30s) begins: Sat Nov 21 20:00:03 UTC 2020
Warmup (30s) ends: Sat Nov 21 20:00:39 UTC 2020
Warmup (30s) result: 295.72 ops/m
Iteration 1 (60s) begins: Sat Nov 21 20:00:39 UTC 2020
Iteration 1 (60s) ends: Sat Nov 21 20:01:43 UTC 2020
Iteration 1 (60s) result: 301.36 ops/m
Iteration 2 (60s) begins: Sat Nov 21 20:01:43 UTC 2020
Iteration 2 (60s) ends: Sat Nov 21 20:02:47 UTC 2020
Iteration 2 (60s) result: 299.76 ops/m
Iteration 3 (60s) begins: Sat Nov 21 20:02:47 UTC 2020
Iteration 3 (60s) ends: Sat Nov 21 20:03:50 UTC 2020
Iteration 3 (60s) result: 300.54 ops/m
Valid run!
Score on compress: 301.36 ops/m
Results are stored in:
/test/results/SPECjvm2008.052/SPECjvm2008.052.raw
Generating reports in:
/test/results/SPECjvm2008.052
Noncompliant composite result: 301.36 ops/m

Specjvm2008 mpegaudio benchmark

java -jar SPECjvm2008.jar -wt 30s -it 1m -bt 16 -i 3 -ikv -ict mpegaudioSPECjvm2008 Peak
Properties file: none
Benchmarks: mpegaudio
WARNING: Run will not be compliant.
Property specjvm.run.initial.check must be true for publication.
Property specjvm.maxiter must be 1 for publication.
Property specjvm.miniter must be 1 for publication.
Not a compliant sequence of benchmarks for publication.
Property specjvm.run.checksum.validation must be true for publication.
Property specjvm.iteration.time must be at least 240 seconds for publication.
--- --- --- --- --- --- --- --- --- Benchmark: mpegaudio
Run mode: timed run
Test type: multi
Threads: 16
Warmup: 30s
Iterations: 3
Run length: 60s
Warmup (30s) begins: Sat Nov 21 20:04:41 UTC 2020
Warmup (30s) ends: Sat Nov 21 20:05:19 UTC 2020
Warmup (30s) result: 159.18 ops/m
Iteration 1 (60s) begins: Sat Nov 21 20:05:19 UTC 2020
Iteration 1 (60s) ends: Sat Nov 21 20:06:25 UTC 2020
Iteration 1 (60s) result: 173.70 ops/m
Iteration 2 (60s) begins: Sat Nov 21 20:06:25 UTC 2020
Iteration 2 (60s) ends: Sat Nov 21 20:07:31 UTC 2020
Iteration 2 (60s) result: 173.62 ops/m
Iteration 3 (60s) begins: Sat Nov 21 20:07:31 UTC 2020
Iteration 3 (60s) ends: Sat Nov 21 20:08:37 UTC 2020
Iteration 3 (60s) result: 174.33 ops/m
Valid run!
Score on mpegaudio: 174.33 ops/m
Results are stored in:
/test/results/SPECjvm2008.053/SPECjvm2008.053.raw
Generating reports in:
/test/results/SPECjvm2008.053
Noncompliant composite result: 174.33 ops/m

Specjvm2008 crypto.rsa benchmark

java -jar SPECjvm2008.jar -wt 30s -it 1m -bt 16 -i 3 -ikv -ict crypto.rsaSPECjvm2008 Peak
Properties file: none
Benchmarks: crypto.rsa
WARNING: Run will not be compliant.
Property specjvm.run.initial.check must be true for publication.
Property specjvm.maxiter must be 1 for publication.
Property specjvm.miniter must be 1 for publication.
Not a compliant sequence of benchmarks for publication.
Property specjvm.run.checksum.validation must be true for publication.
Property specjvm.iteration.time must be at least 240 seconds for publication.
--- --- --- --- --- --- --- --- --- Benchmark: crypto.rsa
Run mode: timed run
Test type: multi
Threads: 16
Warmup: 30s
Iterations: 3
Run length: 60s
Warmup (30s) begins: Sat Nov 21 20:09:33 UTC 2020
Warmup (30s) ends: Sat Nov 21 20:10:03 UTC 2020
Warmup (30s) result: 1790.80 ops/m
Iteration 1 (60s) begins: Sat Nov 21 20:10:04 UTC 2020
Iteration 1 (60s) ends: Sat Nov 21 20:11:04 UTC 2020
Iteration 1 (60s) result: 1817.60 ops/m
Iteration 2 (60s) begins: Sat Nov 21 20:11:04 UTC 2020
Iteration 2 (60s) ends: Sat Nov 21 20:12:05 UTC 2020
Iteration 2 (60s) result: 1814.64 ops/m
Iteration 3 (60s) begins: Sat Nov 21 20:12:05 UTC 2020
Iteration 3 (60s) ends: Sat Nov 21 20:13:06 UTC 2020
Iteration 3 (60s) result: 1814.38 ops/m
Valid run!
Score on crypto.rsa: 1817.60 ops/m
Results are stored in:
/test/results/SPECjvm2008.055/SPECjvm2008.055.raw
Generating reports in:
/test/results/SPECjvm2008.055
Noncompliant composite result: 1817.6 ops/m

Kernel 5.10.0-rc4 without mitigations (mitigations=off in cmdline)

Built from https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git, commit 27bba9c5 (pre 5.10.0-rc5).

Linux debian-blackbird 5.10.0-rc4 #3 SMP Sat Nov 21 11:56:17 -03 2020 ppc64le GNU/Linux❯ cat /proc/cmdline
root=UUID=cc48bfea-adf6-4825-9908-d28de9b9d420 ro mitigations=off

Mitigations:

❯ fgrep -r '' /sys/devices/system/cpu/vulnerabilities
/sys/devices/system/cpu/vulnerabilities/spectre_v2:Mitigation: Indirect branch serialisation (kernel only)
/sys/devices/system/cpu/vulnerabilities/itlb_multihit:Not affected
/sys/devices/system/cpu/vulnerabilities/mds:Not affected
/sys/devices/system/cpu/vulnerabilities/l1tf:Vulnerable: L1D private per thread
/sys/devices/system/cpu/vulnerabilities/spec_store_bypass:Vulnerable
/sys/devices/system/cpu/vulnerabilities/tsx_async_abort:Not affected
/sys/devices/system/cpu/vulnerabilities/spectre_v1:Vulnerable, ori31 speculation barrier enabled
/sys/devices/system/cpu/vulnerabilities/srbds:Not affected
/sys/devices/system/cpu/vulnerabilities/meltdown:Vulnerable: L1D private per thread

7Zip benchmark

❯ 7z b7-Zip 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21
p7zip Version 16.02 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,64 bits,16 CPUs LE)
LE
CPU Freq: 1789 1879 1874 1878 1873 1879 1876 1875 1875
RAM size: 7656 MB, # CPU hardware threads: 16
RAM usage: 3530 MB, # Benchmark threads: 16
Compressing | Decompressing
Dict Speed Usage R/U Rating | Speed Usage R/U Rating
KiB/s % MIPS MIPS | KiB/s % MIPS MIPS
22: 22487 1182 1851 21876 | 302309 1584 1628 25784
23: 22289 1221 1860 22710 | 297122 1578 1629 25707
24: 22095 1268 1874 23758 | 292377 1595 1610 25663
| 25: 21879 1294 1930 24981 | 286932 1587 1609 25536 |
| ---------------------------------- | -------------------------- |
| Avr: 1241 1879 23331 | 1586 1619 25673 |
Tot: 1414 1749 24502

Java SPECjvm2008 running on OpenJDK 8u181 in a Docker container

Docker command: docker run -it --rm -v $(pwd):/test openjdk:8u181-jdk-stretch bash

Specjvm2008 compress benchmark

java -jar SPECjvm2008.jar -wt 30s -it 1m -bt 16 -i 3 -ikv -ict compressSPECjvm2008 Peak
Properties file: none
Benchmarks: compress
WARNING: Run will not be compliant.
Property specjvm.run.initial.check must be true for publication.
Property specjvm.maxiter must be 1 for publication.
Property specjvm.miniter must be 1 for publication.
Not a compliant sequence of benchmarks for publication.
Property specjvm.run.checksum.validation must be true for publication.
Property specjvm.iteration.time must be at least 240 seconds for publication.
--- --- --- --- --- --- --- --- --- Benchmark: compress
Run mode: timed run
Test type: multi
Threads: 16
Warmup: 30s
Iterations: 3
Run length: 60s
Warmup (30s) begins: Sat Nov 21 20:19:02 UTC 2020
Warmup (30s) ends: Sat Nov 21 20:19:37 UTC 2020
Warmup (30s) result: 293.42 ops/m
Iteration 1 (60s) begins: Sat Nov 21 20:19:37 UTC 2020
Iteration 1 (60s) ends: Sat Nov 21 20:20:43 UTC 2020
Iteration 1 (60s) result: 294.49 ops/m
Iteration 2 (60s) begins: Sat Nov 21 20:20:43 UTC 2020
Iteration 2 (60s) ends: Sat Nov 21 20:21:47 UTC 2020
Iteration 2 (60s) result: 298.58 ops/m
Iteration 3 (60s) begins: Sat Nov 21 20:21:47 UTC 2020
Iteration 3 (60s) ends: Sat Nov 21 20:22:52 UTC 2020
Iteration 3 (60s) result: 296.88 ops/m
Valid run!
Score on compress: 298.58 ops/m
Results are stored in:
/test/java/SPECjvm2008/results/SPECjvm2008.056/SPECjvm2008.056.raw
Generating reports in:
/test/java/SPECjvm2008/results/SPECjvm2008.056
Noncompliant composite result: 298.58 ops/m

Specjvm2008 mpegaudio benchmark

java -jar SPECjvm2008.jar -wt 30s -it 1m -bt 16 -i 3 -ikv -ict mpegaudioSPECjvm2008 Peak
Properties file: none
Benchmarks: mpegaudio
WARNING: Run will not be compliant.
Property specjvm.run.initial.check must be true for publication.
Property specjvm.maxiter must be 1 for publication.
Property specjvm.miniter must be 1 for publication.
Not a compliant sequence of benchmarks for publication.
Property specjvm.run.checksum.validation must be true for publication.
Property specjvm.iteration.time must be at least 240 seconds for publication.
--- --- --- --- --- --- --- --- --- Benchmark: mpegaudio
Run mode: timed run
Test type: multi
Threads: 16
Warmup: 30s
Iterations: 3
Run length: 60s
Warmup (30s) begins: Sat Nov 21 20:24:16 UTC 2020
Warmup (30s) ends: Sat Nov 21 20:24:57 UTC 2020
Warmup (30s) result: 161.47 ops/m
Iteration 1 (60s) begins: Sat Nov 21 20:24:57 UTC 2020
Iteration 1 (60s) ends: Sat Nov 21 20:26:03 UTC 2020
Iteration 1 (60s) result: 174.21 ops/m
Iteration 2 (60s) begins: Sat Nov 21 20:26:03 UTC 2020
Iteration 2 (60s) ends: Sat Nov 21 20:27:09 UTC 2020
Iteration 2 (60s) result: 174.58 ops/m
Iteration 3 (60s) begins: Sat Nov 21 20:27:09 UTC 2020
Iteration 3 (60s) ends: Sat Nov 21 20:28:15 UTC 2020
Iteration 3 (60s) result: 174.68 ops/m
Valid run!
Score on mpegaudio: 174.68 ops/m
Results are stored in:
/test/java/SPECjvm2008/results/SPECjvm2008.057/SPECjvm2008.057.raw
Generating reports in:
/test/java/SPECjvm2008/results/SPECjvm2008.057
Noncompliant composite result: 174.68 ops/m

Specjvm2008 crypto.rsa benchmark

java -jar SPECjvm2008.jar -wt 30s -it 1m -bt 16 -i 3 -ikv -ict crypto.rsaSPECjvm2008 Peak
Properties file: none
Benchmarks: crypto.rsa
WARNING: Run will not be compliant.
Property specjvm.run.initial.check must be true for publication.
Property specjvm.maxiter must be 1 for publication.
Property specjvm.miniter must be 1 for publication.
Not a compliant sequence of benchmarks for publication.
Property specjvm.run.checksum.validation must be true for publication.
Property specjvm.iteration.time must be at least 240 seconds for publication.
--- --- --- --- --- --- --- --- --- Benchmark: crypto.rsa
Run mode: timed run
Test type: multi
Threads: 16
Warmup: 30s
Iterations: 3
Run length: 60s
Warmup (30s) begins: Sat Nov 21 21:35:22 UTC 2020
Warmup (30s) ends: Sat Nov 21 21:35:53 UTC 2020
Warmup (30s) result: 1789.03 ops/m
Iteration 1 (60s) begins: Sat Nov 21 21:35:53 UTC 2020
Iteration 1 (60s) ends: Sat Nov 21 21:36:54 UTC 2020
Iteration 1 (60s) result: 1813.11 ops/m
Iteration 2 (60s) begins: Sat Nov 21 21:36:54 UTC 2020
Iteration 2 (60s) ends: Sat Nov 21 21:37:55 UTC 2020
Iteration 2 (60s) result: 1814.90 ops/m
Iteration 3 (60s) begins: Sat Nov 21 21:37:55 UTC 2020
Iteration 3 (60s) ends: Sat Nov 21 21:38:56 UTC 2020
Iteration 3 (60s) result: 1815.36 ops/m
Valid run!
Score on crypto.rsa: 1815.36 ops/m
Results are stored in:
/test/java/SPECjvm2008/results/SPECjvm2008.061/SPECjvm2008.061.raw
Generating reports in:
/test/java/SPECjvm2008/results/SPECjvm2008.061
Noncompliant composite result: 1815.36 ops/m

Feedback

Let me know your feedback on Twitter @carlosedp. Want me to test other benchmarks? Have additional points to add? Drop me a message.

Writing everything cloud and all the tech behind it. If you like my projects and would like to support me, check my Patreon on https://www.patreon.com/carlosedp