Benchmarks

Mac OS

Benchmark: LoTTE Lifestyle 40k

Details: - 40k documents - Dev split - Search data

Hardware: - M1 Macbook Air

colbert:

Average search latency: 0.03s
Median search latency: 0.03s
95th percentile search latency: 0.03s
99th percentile search latency: 0.06s

lintdb:

Average search latency: 0.02s
Median search latency: 0.02s
95th percentile search latency: 0.02s
99th percentile search latency: 0.02s

Linux

Benchmark: LoTTE Lifestyle 40k

Details: - 40k documents - Dev split - Search data

Hardware: Intel(R) Xeon(R) CPU E5-2690 v3 @ 2.60GHz

2024-04-29T03:30:14+00:00
Running ./build_benchmarks/benchmarks/bench_lintdb
Run on (48 X 3500 MHz CPU s)
CPU Caches:
L1 Data 32 KiB (x24)
L1 Instruction 32 KiB (x24)
L2 Unified 256 KiB (x24)
L3 Unified 30720 KiB (x2)
Load Average: 0.10, 2.13, 4.10
***WARNING*** CPU scaling is enabled, the benchmark real time measurements may be noisy and will incur extra overhead.
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20240429 03:30:14.280722 266773 index.cpp:40] loading LintDB from path: experiments/py_index_bench_colbert-lifestyle-2024-04-03
I20240429 03:30:14.675765 266773 index.cpp:40] loading LintDB from path: experiments/py_index_bench_colbert-lifestyle-2024-04-03
I20240429 03:30:15.496747 266773 index.cpp:40] loading LintDB from path: experiments/py_index_bench_colbert-lifestyle-2024-04-03
------------------------------------------------------------------
Benchmark                        Time             CPU   Iterations
------------------------------------------------------------------
BM_lintdb_search_mean         40.2 ms         40.2 ms          212
BM_lintdb_search_median       38.9 ms         38.9 ms          212
BM_lintdb_search_stddev       4.51 ms         4.51 ms          212
BM_lintdb_search_cv          11.21 %         11.21 %           212