diff options
| -rw-r--r-- | BENCHMARKS.md | 251 |
1 files changed, 29 insertions, 222 deletions
diff --git a/BENCHMARKS.md b/BENCHMARKS.md index 5d28f4e..0cbace7 100644 --- a/BENCHMARKS.md +++ b/BENCHMARKS.md @@ -10,231 +10,38 @@ Each benchmark: 2s warmup, 5s measurement window. - **Babashka**: 1.12.212 - **Jank**: 0.1 (Apr 2026) -## JVM (Clojure) - -### v1.3.5 — first-match-wins, linear scan with per-request regex - -| Scenario | Ops/sec | ns/op | -|---|---:|---:| -| Small (5) — literal first | 2,361,346 | 423 | -| Small (5) — literal middle | 1,372,498 | 728 | -| Small (5) — param match | 1,093,600 | 914 | -| Small (5) — nested params | 706,667 | 1,414 | -| Small (5) — wildcard | 1,112,373 | 898 | -| Small (5) — miss (404) | 2,363,419 | 423 | -| Medium (52) — match first | 196,908 | 5,078 | -| Medium (52) — match middle | 47,393 | 21,100 | -| Medium (52) — match last | 19,720 | 50,710 | -| Medium (52) — catch-all wildcard | 17,999 | 55,558 | -| Large (202) — match first | 49,959 | 20,016 | -| Large (202) — match middle | 10,780 | 92,764 | -| Large (202) — match last | 5,090 | 196,464 | -| Large (202) — miss (404) | 10,262 | 97,447 | - -### v2.0.0 — best-match-wins, segment trie with specificity scoring - -| Scenario | Ops/sec | ns/op | -|---|---:|---:| -| Small (5) — literal first | 9,788,503 | 102 | -| Small (5) — literal middle | 4,704,213 | 212 | -| Small (5) — param match | 2,296,556 | 435 | -| Small (5) — nested params | 1,650,276 | 605 | -| Small (5) — wildcard | 1,833,884 | 545 | -| Small (5) — miss (404) | 5,957,701 | 167 | -| Medium (52) — match first | 8,757,566 | 114 | -| Medium (52) — match middle | 1,836,957 | 544 | -| Medium (52) — match last | 1,812,725 | 551 | -| Medium (52) — catch-all wildcard | 2,504,034 | 399 | -| Large (202) — match first | 8,090,398 | 123 | -| Large (202) — match middle | 1,764,621 | 566 | -| Large (202) — match last | 1,683,626 | 593 | -| Large (202) — miss (404) | 3,541,419 | 282 | - -### JVM Comparison - -| Scenario | v1.3.5 ops/s | v2.0.0 ops/s | Speedup | -|---|---:|---:|---:| -| Small (5) — literal first | 2,361,346 | 9,788,503 | 4.1x | -| Small (5) — literal middle | 1,372,498 | 4,704,213 | 3.4x | -| Small (5) — param match | 1,093,600 | 2,296,556 | 2.1x | -| Small (5) — nested params | 706,667 | 1,650,276 | 2.3x | -| Small (5) — wildcard | 1,112,373 | 1,833,884 | 1.6x | -| Small (5) — miss (404) | 2,363,419 | 5,957,701 | 2.5x | -| Medium (52) — match first | 196,908 | 8,757,566 | 44x | -| Medium (52) — match middle | 47,393 | 1,836,957 | 39x | -| Medium (52) — match last | 19,720 | 1,812,725 | 92x | -| Medium (52) — catch-all wildcard | 17,999 | 2,504,034 | 139x | -| Large (202) — match first | 49,959 | 8,090,398 | 162x | -| Large (202) — match middle | 10,780 | 1,764,621 | 164x | -| Large (202) — match last | 5,090 | 1,683,626 | 331x | -| Large (202) — miss (404) | 10,262 | 3,541,419 | 345x | - -## ClojureScript (Node.js) - -### v1.3.5 - -| Scenario | Ops/sec | ns/op | -|---|---:|---:| -| Small (5) — literal first | 1,403,030 | 712 | -| Small (5) — literal middle | 543,057 | 1,841 | -| Small (5) — param match | 179,074 | 5,584 | -| Small (5) — nested params | 169,373 | 5,904 | -| Small (5) — wildcard | 167,200 | 5,980 | -| Small (5) — miss (404) | 170,835 | 5,853 | -| Medium (52) — match first | 27,378 | 36,524 | -| Medium (52) — match middle | 25,216 | 39,656 | -| Medium (52) — match last | 15,788 | 63,335 | -| Medium (52) — catch-all wildcard | 16,028 | 62,389 | -| Large (202) — match first | 27,263 | 36,678 | -| Large (202) — match middle | 6,449 | 155,043 | -| Large (202) — match last | 4,090 | 244,489 | -| Large (202) — miss (404) | 4,028 | 248,201 | - -### v2.0.0 - -| Scenario | Ops/sec | ns/op | -|---|---:|---:| -| Small (5) — literal first | 1,273,187 | 785 | -| Small (5) — literal middle | 849,002 | 1,177 | -| Small (5) — param match | 468,241 | 2,135 | -| Small (5) — nested params | 335,675 | 2,979 | -| Small (5) — wildcard | 370,342 | 2,700 | -| Small (5) — miss (404) | 1,104,045 | 905 | -| Medium (52) — match first | 1,100,616 | 908 | -| Medium (52) — match middle | 361,869 | 2,763 | -| Medium (52) — match last | 358,480 | 2,789 | -| Medium (52) — catch-all wildcard | 506,195 | 1,975 | -| Large (202) — match first | 1,028,507 | 972 | -| Large (202) — match middle | 349,154 | 2,864 | -| Large (202) — match last | 345,575 | 2,893 | -| Large (202) — miss (404) | 671,639 | 1,488 | - -### ClojureScript Comparison - -| Scenario | v1.3.5 ops/s | v2.0.0 ops/s | Speedup | -|---|---:|---:|---:| -| Small (5) — literal first | 1,403,030 | 1,273,187 | 0.9x | -| Small (5) — literal middle | 543,057 | 849,002 | 1.6x | -| Small (5) — param match | 179,074 | 468,241 | 2.6x | -| Small (5) — nested params | 169,373 | 335,675 | 2.0x | -| Small (5) — wildcard | 167,200 | 370,342 | 2.2x | -| Small (5) — miss (404) | 170,835 | 1,104,045 | 6.5x | -| Medium (52) — match first | 27,378 | 1,100,616 | 40x | -| Medium (52) — match middle | 25,216 | 361,869 | 14x | -| Medium (52) — match last | 15,788 | 358,480 | 23x | -| Medium (52) — catch-all wildcard | 16,028 | 506,195 | 32x | -| Large (202) — match first | 27,263 | 1,028,507 | 38x | -| Large (202) — match middle | 6,449 | 349,154 | 54x | -| Large (202) — match last | 4,090 | 345,575 | 84x | -| Large (202) — miss (404) | 4,028 | 671,639 | 167x | - -## Babashka - -### v1.3.5 - -| Scenario | Ops/sec | ns/op | -|---|---:|---:| -| Small (5) — literal first | 178,061 | 5,616 | -| Small (5) — literal middle | 178,653 | 5,597 | -| Small (5) — param match | 125,608 | 7,961 | -| Small (5) — nested params | 121,220 | 8,249 | -| Small (5) — wildcard | 121,145 | 8,254 | -| Small (5) — miss (404) | 172,150 | 5,808 | -| Medium (52) — match first | 27,824 | 35,939 | -| Medium (52) — match middle | 25,177 | 39,718 | -| Medium (52) — match last | 15,973 | 62,604 | -| Medium (52) — catch-all wildcard | 16,087 | 62,161 | -| Large (202) — match first | 27,853 | 35,901 | -| Large (202) — match middle | 6,631 | 150,790 | -| Large (202) — match last | 4,157 | 240,542 | -| Large (202) — miss (404) | 4,206 | 237,747 | - -### v2.0.0 - -| Scenario | Ops/sec | ns/op | -|---|---:|---:| -| Small (5) — literal first | 842,584 | 1,186 | -| Small (5) — literal middle | 540,559 | 1,849 | -| Small (5) — param match | 321,034 | 3,114 | -| Small (5) — nested params | 241,899 | 4,133 | -| Small (5) — wildcard | 324,357 | 3,083 | -| Small (5) — miss (404) | 1,107,631 | 902 | -| Medium (52) — match first | 903,709 | 1,106 | -| Medium (52) — match middle | 285,129 | 3,507 | -| Medium (52) — match last | 292,190 | 3,422 | -| Medium (52) — catch-all wildcard | 424,338 | 2,356 | -| Large (202) — match first | 888,860 | 1,125 | -| Large (202) — match middle | 290,947 | 3,437 | -| Large (202) — match last | 290,664 | 3,440 | -| Large (202) — miss (404) | 767,125 | 1,303 | - -### Babashka Comparison - -| Scenario | v1.3.5 ops/s | v2.0.0 ops/s | Speedup | -|---|---:|---:|---:| -| Small (5) — literal first | 178,061 | 842,584 | 4.7x | -| Small (5) — literal middle | 178,653 | 540,559 | 3.0x | -| Small (5) — param match | 125,608 | 321,034 | 2.6x | -| Small (5) — nested params | 121,220 | 241,899 | 2.0x | -| Small (5) — wildcard | 121,145 | 324,357 | 2.7x | -| Small (5) — miss (404) | 172,150 | 1,107,631 | 6.4x | -| Medium (52) — match first | 27,824 | 903,709 | 32x | -| Medium (52) — match middle | 25,177 | 285,129 | 11x | -| Medium (52) — match last | 15,973 | 292,190 | 18x | -| Medium (52) — catch-all wildcard | 16,087 | 424,338 | 26x | -| Large (202) — match first | 27,853 | 888,860 | 32x | -| Large (202) — match middle | 6,631 | 290,947 | 44x | -| Large (202) — match last | 4,157 | 290,664 | 70x | -| Large (202) — miss (404) | 4,206 | 767,125 | 182x | - -## Jank - -### v2.1.0 - -| Scenario | Ops/sec | ns/op | -|---|---:|---:| -| Small (5) — literal first | 962,079 | 1,039 | -| Small (5) — literal middle | 442,877 | 2,257 | -| Small (5) — param match | 223,006 | 4,484 | -| Small (5) — nested params | 146,768 | 6,813 | -| Small (5) — wildcard | 182,129 | 5,490 | -| Small (5) — miss (404) | 655,948 | 1,524 | -| Medium (52) — match first | 899,823 | 1,111 | -| Medium (52) — match middle | 181,990 | 5,494 | -| Medium (52) — match last | 181,771 | 5,501 | -| Medium (52) — catch-all wildcard | 272,411 | 3,670 | -| Large (202) — match first | 875,741 | 1,141 | -| Large (202) — match middle | 178,075 | 5,615 | -| Large (202) — match last | 176,088 | 5,678 | -| Large (202) — miss (404) | 482,470 | 2,072 | +## Runtime Comparison (Current Versions) + +Current versions: + +- **JVM (Clojure)**: v2.1.1 +- **ClojureScript (Node.js)**: v2.1.1 +- **Babashka**: v2.1.1 +- **Jank**: v2.1.1 + +| Scenario | JVM (Clojure) ops/s | ClojureScript (Node.js) ops/s | Babashka ops/s | Jank ops/s | +|---|---:|---:|---:|---:| +| Small (5) - literal first | 5,751,517 | 826,109 | 561,707 | 632,139 | +| Small (5) - literal middle | 2,236,756 | 458,372 | 328,534 | 293,751 | +| Small (5) - param match | 1,138,681 | 244,455 | 194,415 | 147,771 | +| Small (5) - nested params | 725,794 | 159,997 | 136,953 | 95,436 | +| Small (5) - wildcard | 1,038,987 | 222,914 | 205,294 | 121,320 | +| Small (5) - miss (404) | 3,507,344 | 696,492 | 611,017 | 452,239 | +| Medium (52) - match first | 5,464,333 | 729,185 | 559,724 | 615,761 | +| Medium (52) - match middle | 947,366 | 198,538 | 167,396 | 124,459 | +| Medium (52) - match last | 943,909 | 199,165 | 168,330 | 126,711 | +| Medium (52) - catch-all wildcard | 1,611,263 | 339,559 | 274,306 | 191,428 | +| Large (202) - match first | 5,086,938 | 668,097 | 553,183 | 606,074 | +| Large (202) - match middle | 931,339 | 195,975 | 168,974 | 125,689 | +| Large (202) - match last | 925,180 | 195,752 | 168,347 | 124,528 | +| Large (202) - miss (404) | 2,278,714 | 443,050 | 458,547 | 338,429 | ## Analysis -### JVM (Clojure) -- **Small route sets**: 1.6–4.1x faster -- **Medium route sets**: 39–139x faster -- **Large route sets**: 162–345x faster -- Peak throughput: ~9.8M ops/sec (literal match) - -### ClojureScript (Node.js) -- **Small route sets**: 0.9–6.5x faster (literal-first is within noise; params, wildcards, and misses see large gains) -- **Medium route sets**: 14–40x faster -- **Large route sets**: 38–167x faster -- Peak throughput: ~1.3M ops/sec (literal match) - -### Babashka -- **Small route sets**: 2.0–6.4x faster -- **Medium route sets**: 11–32x faster -- **Large route sets**: 32–182x faster -- Peak throughput: ~1.1M ops/sec (miss/404 — fast trie rejection) - -### Jank -- Peak throughput: ~962K ops/sec (literal match) -- Performance is now close to Babashka peak throughput on this benchmark suite - -### Key insight - -v2.0 performance is nearly independent of route count across all runtimes. Matching is O(path depth) via the trie, not O(route count) via linear scan. The improvement is most dramatic for large route sets where v1.x's linear scan with per-request regex compilation becomes the dominant bottleneck. +- **JVM (Clojure)** leads in all scenarios in this benchmark set, peaking at ~5.8M ops/sec. +- **ClojureScript (Node.js)**, **Babashka**, and **Jank** are in the same general tier, with ClojureScript typically leading and Jank usually close behind. +- **Babashka** and **Jank** are very close on several large-route scenarios, with Babashka slightly ahead on large miss/404. +- Across runtimes, throughput stays relatively stable as route count grows, consistent with trie-based matching by path depth. ### Running benchmarks |