January 12, 2021

Menagerie Part 1B: Speed Comparisons

Menagerie Part 1B: Speed Comparisons
  • This is part of my Database Menagerie series, where we setup a geographically dispersible database.  
  • Go HERE  for the introduction and table of contents.  
  • This is a work-in-progress: last change: 01/12/2021

Below are my speed tests, based on all having the exact same SSD and all having 8GB of RAM (except 4GB for Pi).

Devices Tested

Device CPU RAM Disk OS Notes
NUC/J4005 Intel Celeron J4005
* 2.7GHz
* 2-core
* 2-thread
8GB 250GB SSD
* Crucial MX500
Ubuntu 20.10 64-bit Intel NUC7CJYH
m710qTiny Intel Core i5-6500T
* 3.1GHz
* 4-core
* 4-thread
8GB 250GB SSD
* Crucial MX500
Ubuntu 20.10 64-bit Lenovo ThinkCentre Tiny
RPi4 ARM BCM2835
* 1.5GHz
* 4-core
* 4-thread
4GB 250GB SSD
* Crucial MX500
Ubuntu 20.10 64-bit
RPi4/DeskPi same as above same as above same as above same as above SSD via DeskPi's non-UASP SATA
RPi4/X825 same as above same as above same as above same as above SSD via Geekworm's UASP SATA (X825)
WSL2ryzen AMD Ryzen 5 2600X
* 3.8GHz
* 6-core
* 12-thread
16GB NVME SSD Ubuntu 20.04 64-bit Just used for a comparison; WSL2 on Windows 10
WSL2work Intel Core i5-7400
* 3.0GHz
* 4-core
* 4-thread
8GB 1TB HDD
slow disk
with PrimoCache added
Ubuntu 20.04 64-bit Just used for a comparison; WSL2 on Windows 10

sudo curl https://raw.githubusercontent.com/TheRemote/PiBenchmarks/master/Storage.sh | sudo bash
Winner Device Result Notes
2 NUC/J4005 20,111 SSD, dual core/no hyperthreads
1 m710qTiny 30,872 SSD, quad core
4 RPi4/X825 8,332
tbd RPi4/DeskPi
5 WSL2work 6,681 WSL2 on slow HDD w/PrimoCache

Cockroach CPU Test

cockroach systembench cpu

Winner Device Lowest Highest Notes
tbd RPi4
2 NUC/J4005 100,252,286 108,556,537 seems high - dual core
3 m710qTiny 71,621,537 71,980,640 seems low - quad core
1 WSL2ryzen 122,166,460 135,963,060
4 WSL2work 67,390,385 74,515,442

Cockroach DISK Test

cockroach systembench seqwrite

Winner Device Lowest
Ops/Sec
Highest
Ops/Sec
Notes
tbd RPi4/X825
tbd RPi4/DeskPi
2 NUC/J4005 2,751 3,695
1 m710qTiny 3,326 5,565
3 WSL2ryzen 1,208 1,639 fast NVMe but WSL2
4 WSL2work 144 336 slow disk but primoCached

hdparm DISK Test

sudo hdparm -tT /dev/sda

Winner Device Lowest Cached
MB/sec
Highest Buffered
MB/sec
RPi4/X825 814.15 358.75
RPi4/DeskPi
NUC/J4005 4,985.69 536.29
1 B m710qTiny 13,004.38 523.41
1 C WSL2ryzen 11,146.16 1,435.19
WSL2work 11,844.16 802.36

sysbench CPU test

sudo apt install sysbench
sysbench cpu --threads=4 --cpu-max-prime=9999 run

Winner Device Events/sec Notes
2 RPi4 5,959.39 seems odd to be 2nd fastest
5 NUC/J4005 3,123.12
4 m710qTiny 3,956.56
1 WSL2ryzen 6,372.57 OK, should be the fastest
3 WSL2work 4,028.03

Cockroach TPC-C benchmark

SOURCE: laptop test from https://www.cockroachlabs.com/docs/v20.2/performance-benchmarking-with-tpcc-local.html

cockroach workload init tpcc --warehouses=10 'postgresql://root@IPADDRESS:26257?sslmode=disable'
# may take a few minutes to load

cockroach workload run tpcc --warehouses=10 --ramp=3m --duration=10m 'postgresql://root@IPADDRESS:26257?sslmode=disable'
# runs for ten minutes

# review the final output line for speed rating 
## on 3 nodes, with 1 node also running the SQL, results were
### tpmC was 123.7, p90 was 35.7, efficiency was 96.2%
### p90 should be in tens of ms per Cockroach (TPC-C requirement is in the order of seconds)

Cockroach TPC-C benchmark results

not the best test since the test SQL commands were running on one of the nodes and not load-balanced
except the last test was run on a non-node (but not load-balanced), but the results are slightly lower tpmC

# on ltrcr01 (3 nodes)
_elapsed_______tpmC____efc__avg(ms)__p50(ms)__p90(ms)__p95(ms)__p99(ms)_pMax(ms)
  600.0s      123.7  96.2%     30.9     31.5     35.7     37.7     50.3     79.7

# on ltcr02 (3 nodes)
_elapsed_______tpmC____efc__avg(ms)__p50(ms)__p90(ms)__p95(ms)__p99(ms)_pMax(ms)
  600.0s      120.9  94.0%     39.4     39.8     44.0     46.1     52.4     71.3```

# on incr01 (3 nodes)
_elapsed_______tpmC____efc__avg(ms)__p50(ms)__p90(ms)__p95(ms)__p99(ms)_pMax(ms)
  600.0s      121.5  94.5%     64.9     62.9     79.7     92.3    151.0    369.1

# on ltcr02 while ltcr01 was runnng movr  (3 nodes)
_elapsed_______tpmC____efc__avg(ms)__p50(ms)__p90(ms)__p95(ms)__p99(ms)_pMax(ms)
  600.0s      123.1  95.7%     32.3     33.6     37.7     39.8     52.4     79.7

# on ltcr02 connected to ltcr01 (5 nodes)
_elapsed_______tpmC____efc__avg(ms)__p50(ms)__p90(ms)__p95(ms)__p99(ms)_pMax(ms)
  600.0s      118.6  92.2%    710.6    704.6    805.3    906.0   1208.0   1610.6

# on separate WSL2 cockroach connected to incr03 (5 nodes)
_elapsed_______tpmC____efc__avg(ms)__p50(ms)__p90(ms)__p95(ms)__p99(ms)_pMax(ms)
  600.0s      119.0  92.5%    622.2    604.0    771.8   1208.0   1476.4   2415.9

# same as above - second run
_elapsed_______tpmC____efc__avg(ms)__p50(ms)__p90(ms)__p95(ms)__p99(ms)_pMax(ms)
  600.0s      120.8  93.9%    578.1    570.4    704.6    805.3   1006.6   2013.3

Cockroach YCSB Benchmark results

First run (5 nodes, running command on a non-node cockroach connected to incr03)

_elapsed___errors_____ops(total)___ops/sec(cum)__avg(ms)__p50(ms)__p95(ms)__p99(ms)_pMax(ms)__total
  600.0s        0         104138          173.6     43.6     44.0     58.7     83.9    436.2  read

_elapsed___errors_____ops(total)___ops/sec(cum)__avg(ms)__p50(ms)__p95(ms)__p99(ms)_pMax(ms)__total
  600.0s        0           5466            9.1     46.6     46.1     62.9     92.3    318.8  update

_elapsed___errors_____ops(total)___ops/sec(cum)__avg(ms)__p50(ms)__p95(ms)__p99(ms)_pMax(ms)__result
  600.0s        0         109604          182.7     43.8     44.0     58.7     83.9    436.2

Second test running two simultaneously, both on a non-node-cockroach (to incr03 and ltcr01)

_elapsed___errors_____ops(total)___ops/sec(cum)__avg(ms)__p50(ms)__p95(ms)__p99(ms)_pMax(ms)__total
  600.0s        0          97170          161.9     46.8     41.9     65.0    285.2    570.4  read

_elapsed___errors_____ops(total)___ops/sec(cum)__avg(ms)__p50(ms)__p95(ms)__p99(ms)_pMax(ms)__total
  600.0s        0           5062            8.4     49.0     46.1     71.3    125.8    402.7  update

_elapsed___errors_____ops(total)___ops/sec(cum)__avg(ms)__p50(ms)__p95(ms)__p99(ms)_pMax(ms)__result
  600.0s        0         102232          170.4     46.9     44.0     65.0    285.2    570.4

James Chambers complete results

m710qTiny complete test results ------------------------------------------------------------

     Category                  Test                      Result
HDParm                    Disk Read                 467.95 MB/s
HDParm                    Cached Disk Read          389.39 MB/s
DD                        Disk Write                357 MB/s
FIO                       4k random read            95700 IOPS (382803 KB/s)
FIO                       4k random write           68494 IOPS (273979 KB/s)
IOZone                    4k read                   133493 KB/s
IOZone                    4k write                  85587 KB/s
IOZone                    4k random read            37179 KB/s
IOZone                    4k random write           120619 KB/s

                          Score: 30072

Intel NUC J4005 complete test results -------------------------------------------------------

     Category                  Test                      Result     
HDParm                    Disk Read                 473.21 MB/s
HDParm                    Cached Disk Read          415.99 MB/s
DD                        Disk Write                226 MB/s
FIO                       4k random read            46972 IOPS (187889 KB/s)
FIO                       4k random write           28643 IOPS (114573 KB/s)
IOZone                    4k read                   106494 KB/s
IOZone                    4k write                  74393 KB/s
IOZone                    4k random read            40467 KB/s
IOZone                    4k random write           96264 KB/s

                          Score: 20111

RPi4 complete test results ---------------------------------------------------------------

     Category                  Test                      Result
HDParm                    Disk Read                 292.60 MB/s
HDParm                    Cached Disk Read          198.77 MB/s
DD                        Disk Write                116 MB/s
FIO                       4k random read            13553 IOPS (54215 KB/s)
FIO                       4k random write           16953 IOPS (67814 KB/s)
IOZone                    4k read                   29304 KB/s
IOZone                    4k write                  24926 KB/s
IOZone                    4k random read            18218 KB/s
IOZone                    4k random write           28467 KB/s

                          Score: 7542

WSL2work complete test results -----------------------------------------------------------

     Category                  Test                      Result
HDParm                    Disk Read                 1094.48 MB/s
HDParm                    Cached Disk Read          839.91 MB/s
DD                        Disk Write                633 MB/s
FIO                       4k random read            301 IOPS (1205 KB/s)
FIO                       4k random write           172 IOPS (691 KB/s)
IOZone                    4k read                   3892 KB/s
IOZone                    4k write                  5588 KB/s
IOZone                    4k random read            365 KB/s
IOZone                    4k random write           605 KB/s

                          Score: 6681

WSL2ryzen complete test results -----------------------------------------------------------

     Category                  Test                      Result
HDParm                    Disk Read                 806.49 MB/s
HDParm                    Cached Disk Read          590.01 MB/s
DD                        Disk Write                911 MB/s
FIO                       4k random read            99417 IOPS (397669 KB/s)
FIO                       4k random write           8188 IOPS (32754 KB/s)
IOZone                    4k read                   25815 KB/s
IOZone                    4k write                  23693 KB/s
IOZone                    4k random read            18367 KB/s
IOZone                    4k random write           25564 KB/s

                          Score: 17910

Sidenote: Hardware details can be seen via:

sudo lshw

  • CPU-only specifics via
    sudo lshw -C cpu

Comments powered by Talkyard.

Buy Me A Coffee