In this blog post, we'll dive into a performance comparison between Redis, the well-known in-memory data store, and HPKV, which provides disk persistence with immediate consistency and low latency. We'll look at both single operations and batch operations to understand how these systems perform under different workloads. This performance comparison is meant to be a simple yet fair comparison between the two systems and in near future we will provide more detailed benchmarks with not just Redis but other key-value stores.

Test Environment

The benchmarks were conducted in a controlled environment with the following specifications:

Linux VM with kernel 6.8.0-50
4 CPU cores
8 GB RAM
Host: Apple M3 Max with 64GB RAM

Methodology

For Redis, we followed the official benchmarking methodology as described in the Redis benchmarking guide. The tests were conducted using redis-benchmark.

For HPKV, we used the RIOC benchmarking tool, which was configured to match the Redis benchmark parameters. HPKV also provides a local vectored call interface which can bring the single operation latency close to 250ns range, but since redis is operating as a server and to keep the comparison fair, we used RIOC for this benchmark.

Both systems were running locally, to ensure that the network latency is minimal and the performance is not affected by the network.

NOTE

RIOC is NOT operating over Unix sockets and it's using the whole network stack for this benchmark as opposed to Redis which can operate over Unix sockets when run locally which gives redis a slight edge.

Test Setup

No. of records: 1 million
Value size: 100 bytes
No. of concurrent clients: 50

Performance Results

Let's examine the performance comparison between Redis and HPKV for both single and batch operations:

Redis vs HPKV Performance Benchmark

The graphs above clearly illustrate the performance advantages of HPKV over Redis in both single and batch operations. Let's break down the key findings:

Single Operation Performance

Redis Single Operations

SET: 273,672.69 requests per second (p50=0.095 ms)
GET: 278,164.12 requests per second (p50=0.095 ms)

HPKV Single Operations

INSERT: 1,082,578.12 operations per second
- P50 latency: 11.183 μs
- P95 latency: 46.040 μs
- P99 latency: 217.682 μs

GET: 1,728,939.43 operations per second
- P50 latency: 8.632 μs
- P95 latency: 35.843 μs
- P99 latency: 88.647 μs

DELETE: 935,846.09 operations per second
- P50 latency: 11.470 μs
- P95 latency: 59.797 μs
- P99 latency: 294.093 μs

In single operation mode, HPKV shows significantly higher throughput compared to Redis:

Performance Comparison:

Write operations (SET/INSERT): 4x faster

Read operations (GET): 6.2x faster

Batch Operation Performance

Next, let's examine how both systems perform with batch operations (batch size of 16):

Redis Batch Operations

SET: 2,439,024.50 requests per second (p50=0.263 ms)
GET: 2,932,551.50 requests per second (p50=0.223 ms)

HPKV Batch Operations

INSERT: 6,125,538.03 operations per second
- P50 latency: 0.903 μs
- P95 latency: 12.144 μs
- P99 latency: 84.452 μs

GET: 8,273,300.27 operations per second
- P50 latency: 0.545 μs
- P95 latency: 3.752 μs
- P99 latency: 63.501 μs

DELETE: 5,705,816.00 operations per second
- P50 latency: 0.919 μs
- P95 latency: 10.422 μs
- P99 latency: 72.161 μs

In batch operation mode, both systems show significant performance improvements, but HPKV maintains its performance advantage:

Performance Comparison:

Write operations: 2.5x faster

Read operations: 2.8x faster

Key Observations

Consistent Performance: Both systems show consistent performance with low latency variations, as evidenced by the p50, p95, and p99 latency metrics.
Batch Operation Benefits: Both systems benefit significantly from batch operations, with HPKV showing:
- INSERT performance: 5.7x improvement
- GET performance: 4.8x improvement
- DELETE performance: 6.1x improvement
Latency Characteristics: HPKV shows remarkably low latencies, especially in batch operations, with median latencies under 1 microsecond for all operations.

Conclusion

The benchmark results demonstrate that HPKV provides superior performance compared to Redis in both single and batch operations. This performance advantage is particularly noticeable in single operations, where HPKV shows 4-6x better throughput.

Get started with HPKV today and experience the performance benefits of an in-memory data store with disk persistence.