Benchmark your Cloud Server - CPU

The internet is full of IaaS Providers, everybody has now their own Cloud Solution and they're willing to give you a piece (not for free though).

Making the right choice is not an easy task. They all seem to offer similar products for the same price. If you don't need something fancy, then it's all about your personal choice.

But if you need some heavy horse power, you'd better benchmark the products before going into full production.

In this post we'll test one of the main resources of a system - the CPU.
For the test to be relevant, you need to:
- "buy" the same hardware resources, the CPU in our case
- have the same operating system, kernel and updates
- use the same version and options of the testing tools

For my benchmark, i'll be using two VMs (unfortunately the CPUs differ, but the tests are still very relevant):
- one Debian vmWare hosted on a dedicated server, 1vCPU @ 3.20GHz (Intel(R) Core(TM) i7-3930K CPU @ 3.20GHz)
- one M3.medium Amazon Debian instance, 1vCPU @ 2.50GHz (Intel Xeon E5-2670 Sandy Bridge)
[cc lang='bash']
# vm1
\$ lscpu
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 1
On-line CPU(s) list: 0
Thread(s) per core: 1
Core(s) per socket: 1
Socket(s): 1
NUMA node(s): 1
Vendor ID: GenuineIntel
CPU family: 6
Model: 45
Stepping: 7
CPU MHz: 3200.025
BogoMIPS: 6400.05
Hypervisor vendor: VMware
Virtualization type: full
L1d cache: 32K
L1i cache: 32K
L2 cache: 256K
L3 cache: 12288K
NUMA node0 CPU(s): 0

# vm2
\$ lscpu
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 1
On-line CPU(s) list: 0
Thread(s) per core: 1
Core(s) per socket: 1
Socket(s): 1
NUMA node(s): 1
Vendor ID: GenuineIntel
CPU family: 6
Model: 62
Stepping: 4
CPU MHz: 2500.050
BogoMIPS: 5000.10
Hypervisor vendor: Xen
Virtualization type: full
L1d cache: 32K
L1i cache: 32K
L2 cache: 256K
L3 cache: 25600K
NUMA node0 CPU(s): 0
[/cc]

7z (single threaded use)
[cc lang='bash']
# vm1
\$ apt-get install p7zip-full -y
\$ 7z b

RAM size: 1002 MB, # CPU hardware threads: 1
RAM usage: 419 MB, # Benchmark threads: 1

Dict Compressing | Decompressing
Speed Usage R/U Rating | Speed Usage R/U Rating
KB/s % MIPS MIPS | KB/s % MIPS MIPS

22: 3804 100 3712 3700 | 40065 100 3628 3617
23: 3553 99 3641 3621 | 39572 100 3638 3623
24: 3363 100 3631 3616 | 39105 99 3651 3628
25: 3222 100 3689 3679 | 38397 99 3633 3611

Avr: 100 3668 3654 100 3637 3620
Tot: 100 3653 3637

# vm2
\$ apt-get install p7zip-full -y
\$ 7z b

RAM size: 3750 MB, # CPU hardware threads: 1
RAM usage: 419 MB, # Benchmark threads: 1

Dict Compressing | Decompressing
Speed Usage R/U Rating | Speed Usage R/U Rating
KB/s % MIPS MIPS | KB/s % MIPS MIPS

22: 1557 88 1729 1515 | 16666 87 1721 1504
23: 1517 87 1770 1546 | 16468 87 1735 1507
24: 1404 87 1735 1509 | 16298 88 1718 1512
25: 1397 88 1814 1595 | 15945 88 1712 1499

Avr: 87 1762 1541 87 1722 1506
Tot: 87 1742 1524
[/cc]
-> vm1 is 2x faster than vm2
-> on vm2 we couldn't use 100% of the cpu due to high Steal Time (you can check it with top command while running the benchmark)

sysbench (single threaded use)
[cc lang='bash']
# vm1
\$ apt-get install sysbench -y
\$ sysbench --test=cpu --cpu-max-prime=20000 run
sysbench 0.4.12: multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 1

Doing CPU performance benchmark

Threads started!
Done.

Maximum prime number checked in CPU test: 20000

Test execution summary:
total time: 24.5747s
total number of events: 10000
total time taken by event execution: 24.5737
per-request statistics:
min: 2.40ms
avg: 2.46ms
max: 7.43ms
approx. 95 percentile: 2.46ms

Threads fairness:
events (avg/stddev): 10000.0000/0.00
execution time (avg/stddev): 24.5737/0.00

# vm2
\$ apt-get install sysbench -y
\$ sysbench --test=cpu --cpu-max-prime=20000 run
sysbench 0.4.12: multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 1

Doing CPU performance benchmark

Threads started!
Done.

Maximum prime number checked in CPU test: 20000

Test execution summary:
total time: 59.0544s
total number of events: 10000
total time taken by event execution: 59.0467
per-request statistics:
min: 2.89ms
avg: 5.90ms
max: 19.13ms
approx. 95 percentile: 11.11ms

Threads fairness:
events (avg/stddev): 10000.0000/0.00
execution time (avg/stddev): 59.0467/0.00
[/cc]
-> sysbench confirms that vm1 is 2x faster than vm2

wiki'd

by JoKeru

Benchmark your Cloud Server - CPU

Comments