Commit Overhead vs Throughput

Commit Overhead vs Throughput

Analysis of commit protocol overhead as percentage of total transaction time.

Overhead = (commit_latency / total_latency) × 100

This represents time spent in retries, exponential backoff, and manifest I/O.

Values shown as mean ± standard deviation across seeds.

num_tables = 1

Throughput (c/s)Success Rate (%)Mean Overhead (%)P50 Overhead (%)P95 Overhead (%)P99 Overhead (%)
0.2 ± 0.0100.0 ± 0.00.4 ± 0.00.3 ± 0.00.6 ± 0.00.7 ± 0.0
0.5 ± 0.0100.0 ± 0.00.8 ± 0.00.8 ± 0.01.1 ± 0.01.2 ± 0.0
1.0 ± 0.0100.0 ± 0.01.5 ± 0.01.5 ± 0.01.8 ± 0.02.0 ± 0.1
1.9 ± 0.0100.0 ± 0.02.8 ± 0.12.8 ± 0.13.3 ± 0.13.5 ± 0.1
4.9 ± 0.0100.0 ± 0.06.9 ± 0.06.9 ± 0.07.5 ± 0.17.9 ± 0.1
9.8 ± 0.198.9 ± 0.013.7 ± 0.113.6 ± 0.114.4 ± 0.114.8 ± 0.1
15.1 ± 0.076.3 ± 0.521.1 ± 0.121.1 ± 0.021.6 ± 0.021.8 ± 0.0
17.5 ± 0.035.1 ± 0.124.5 ± 0.024.5 ± 0.024.9 ± 0.025.0 ± 0.0
18.6 ± 0.018.1 ± 0.125.9 ± 0.025.9 ± 0.026.3 ± 0.026.4 ± 0.0

num_tables = 2

Throughput (c/s)Success Rate (%)Mean Overhead (%)P50 Overhead (%)P95 Overhead (%)P99 Overhead (%)
0.2 ± 0.0100.0 ± 0.00.4 ± 0.00.4 ± 0.00.6 ± 0.00.7 ± 0.0
0.5 ± 0.0100.0 ± 0.00.8 ± 0.00.8 ± 0.01.1 ± 0.01.2 ± 0.1
1.0 ± 0.0100.0 ± 0.01.5 ± 0.01.5 ± 0.01.8 ± 0.02.0 ± 0.1
2.0 ± 0.0100.0 ± 0.02.8 ± 0.02.8 ± 0.03.3 ± 0.03.5 ± 0.1
4.9 ± 0.0100.0 ± 0.07.0 ± 0.16.9 ± 0.17.5 ± 0.17.9 ± 0.1
9.9 ± 0.1100.0 ± 0.013.8 ± 0.113.8 ± 0.114.6 ± 0.115.0 ± 0.1
18.9 ± 0.095.3 ± 0.126.2 ± 0.126.2 ± 0.126.9 ± 0.127.3 ± 0.1
26.1 ± 0.052.3 ± 0.336.2 ± 0.036.2 ± 0.036.6 ± 0.036.7 ± 0.0
28.3 ± 0.027.8 ± 0.139.3 ± 0.039.2 ± 0.039.6 ± 0.039.7 ± 0.0

num_tables = 5

Throughput (c/s)Success Rate (%)Mean Overhead (%)P50 Overhead (%)P95 Overhead (%)P99 Overhead (%)
0.2 ± 0.0100.0 ± 0.00.3 ± 0.00.3 ± 0.00.5 ± 0.00.7 ± 0.1
0.5 ± 0.0100.0 ± 0.00.8 ± 0.00.7 ± 0.01.0 ± 0.01.2 ± 0.1
1.0 ± 0.0100.0 ± 0.01.5 ± 0.01.4 ± 0.01.8 ± 0.02.0 ± 0.1
2.0 ± 0.0100.0 ± 0.02.8 ± 0.02.8 ± 0.03.2 ± 0.03.4 ± 0.1
5.0 ± 0.1100.0 ± 0.06.9 ± 0.16.9 ± 0.17.5 ± 0.17.8 ± 0.1
9.8 ± 0.1100.0 ± 0.013.7 ± 0.113.7 ± 0.114.4 ± 0.114.8 ± 0.2
19.7 ± 0.199.8 ± 0.027.3 ± 0.127.3 ± 0.228.2 ± 0.228.6 ± 0.3
30.9 ± 0.162.4 ± 0.142.7 ± 0.042.7 ± 0.043.1 ± 0.043.2 ± 0.0
33.3 ± 0.032.9 ± 0.146.1 ± 0.046.1 ± 0.046.4 ± 0.046.5 ± 0.0

num_tables = 10

Throughput (c/s)Success Rate (%)Mean Overhead (%)P50 Overhead (%)P95 Overhead (%)P99 Overhead (%)
0.2 ± 0.0100.0 ± 0.00.3 ± 0.00.3 ± 0.00.5 ± 0.00.7 ± 0.1
0.5 ± 0.0100.0 ± 0.00.7 ± 0.00.7 ± 0.01.0 ± 0.01.1 ± 0.1
1.0 ± 0.0100.0 ± 0.01.4 ± 0.01.4 ± 0.01.8 ± 0.01.9 ± 0.1
2.0 ± 0.0100.0 ± 0.02.8 ± 0.12.7 ± 0.03.2 ± 0.13.4 ± 0.1
4.9 ± 0.1100.0 ± 0.06.9 ± 0.16.9 ± 0.17.5 ± 0.17.8 ± 0.2
9.9 ± 0.1100.0 ± 0.013.7 ± 0.113.7 ± 0.114.5 ± 0.114.8 ± 0.1
19.7 ± 0.1100.0 ± 0.027.3 ± 0.127.3 ± 0.128.1 ± 0.128.4 ± 0.2
33.4 ± 0.067.9 ± 0.146.2 ± 0.046.2 ± 0.046.6 ± 0.046.8 ± 0.0
36.4 ± 0.036.0 ± 0.150.3 ± 0.050.3 ± 0.050.5 ± 0.050.6 ± 0.0

num_tables = 20

Throughput (c/s)Success Rate (%)Mean Overhead (%)P50 Overhead (%)P95 Overhead (%)P99 Overhead (%)
0.2 ± 0.0100.0 ± 0.00.3 ± 0.00.3 ± 0.00.5 ± 0.00.6 ± 0.0
0.5 ± 0.0100.0 ± 0.00.6 ± 0.00.7 ± 0.01.0 ± 0.01.1 ± 0.1
1.0 ± 0.0100.0 ± 0.01.2 ± 0.01.4 ± 0.01.7 ± 0.01.9 ± 0.1
2.0 ± 0.0100.0 ± 0.02.7 ± 0.12.7 ± 0.13.2 ± 0.13.4 ± 0.0
4.9 ± 0.0100.0 ± 0.06.8 ± 0.16.8 ± 0.17.4 ± 0.17.7 ± 0.1
9.9 ± 0.1100.0 ± 0.013.7 ± 0.213.7 ± 0.114.4 ± 0.214.8 ± 0.2
19.7 ± 0.1100.0 ± 0.027.2 ± 0.127.3 ± 0.228.0 ± 0.128.4 ± 0.2
35.8 ± 0.073.0 ± 0.249.4 ± 0.049.4 ± 0.049.8 ± 0.050.0 ± 0.0
39.3 ± 0.039.2 ± 0.154.2 ± 0.054.2 ± 0.054.5 ± 0.054.6 ± 0.0

num_tables = 50

Throughput (c/s)Success Rate (%)Mean Overhead (%)P50 Overhead (%)P95 Overhead (%)P99 Overhead (%)
0.2 ± 0.0100.0 ± 0.00.3 ± 0.00.3 ± 0.00.6 ± 0.00.7 ± 0.0
0.5 ± 0.0100.0 ± 0.00.5 ± 0.00.7 ± 0.01.0 ± 0.01.1 ± 0.0
1.0 ± 0.0100.0 ± 0.01.0 ± 0.01.3 ± 0.01.7 ± 0.01.9 ± 0.0
2.0 ± 0.0100.0 ± 0.02.2 ± 0.12.7 ± 0.03.1 ± 0.13.3 ± 0.1
4.9 ± 0.0100.0 ± 0.06.5 ± 0.06.7 ± 0.07.3 ± 0.17.6 ± 0.2
9.9 ± 0.1100.0 ± 0.013.4 ± 0.113.6 ± 0.114.2 ± 0.114.6 ± 0.1
19.8 ± 0.1100.0 ± 0.027.0 ± 0.127.1 ± 0.228.0 ± 0.228.4 ± 0.2
38.9 ± 0.179.4 ± 0.353.4 ± 0.153.4 ± 0.153.8 ± 0.154.0 ± 0.1
43.2 ± 0.043.6 ± 0.159.4 ± 0.059.4 ± 0.059.7 ± 0.059.8 ± 0.0

Interpretation

  • Low overhead (<10%): System operating efficiently, minimal contention
  • Medium overhead (10-30%): Moderate contention, acceptable performance
  • High overhead (30-50%): High contention, commit protocol becoming significant
  • Very high overhead (>50%): Commit protocol dominates, system saturated

Values shown as mean ± standard deviation across multiple seeds. At saturation, overhead can exceed 50%, meaning more time is spent retrying commits than executing transactions!