Searching waveforms by shape inside PostgreSQL

With recordings, you often ask: where else did this shape occur, even a little earlier or later? Our first report asks whether PostgreSQL can answer that quickly with a standard vector index, while every hit still points to the exact stored samples. A hit you cannot read back bit for bit is a hint, not evidence.

What we found

• Ordinary vector search asks the wrong question. On 235,000 seismic windows, a zero-lag shortlist of 1,000 recovered only 74% of the lag-tolerant scorer’s top ten. An exact shortlist lost the same matches as an HNSW one, so the candidate metric is the cause, not approximate search.
• Thirteen shifted probes fix it. Each lag term is an inner product with a shifted copy of the query, so 13 shifted queries into one unmodified HNSW index, re-ranked with the unchanged scorer, find the candidates. No training, no per-shift copies. On held-out stations and year, probe search recovered 97.7% of the exact top ten at a p95 of 57 ms, against 740 ms for in-database exact search on the measured build. Zero-lag candidates from the same index recovered 0.639–0.658.
• One contract on both paths. Every hit is a sample interval that reads back the losslessly stored recording bit for bit. Filters and row-level security apply to exact and probe search alike.
• A first relevance check. With a catalogued earthquake in the query window, the bounded-lag top ten held catalogued earthquakes in 0.297 of positions against 0.213 for zero lag on development data, and 0.139 against 0.060 on held-out stations and year, using exact search and the catalogue as a proxy for relevance.
• It has costs. Probe structures take 0.97–1.34× the raw data, and index maintenance wrote 18 times the database growth in write-ahead log with default settings (2.9 times when tuned).

Exact search has become faster since the main comparison: a float32 screen with a rigorous error bound (study E7) cut its p95 latency 3.2–3.7× with bit-for-bit identical lists. Probes stayed 1.6–2.3× faster at p95 on galleries of up to 1.03 million windows.

Where it falls short

• One signal domain. All retrieval results in the report are for seismic data from one network; study N1 has since tested appliance identification from current. Vibration and acoustic data are not measured, and there is no evidence yet that this search is diagnostically relevant for machines.
• No recall guarantee. Recall is measured, not bounded. Per query it fell to 7 of 10 on held-out data.
• Narrow conditions. One laptop-class machine, warm caches, one HNSW configuration. The planned two-hour load run stopped after 111 minutes with an unexplained pause. No claim about cold caches or archives beyond about one million windows.
• Not a new confirmation. The held-out data had been used once before, so the main comparison re-measured them. The 13-probe grid was chosen on development data and explained afterwards.
• Exact search still wins sometimes. Small or selective galleries go to exact search. An exact scan of a copy outside the database was 2.5–3.3× faster than probes at p95, without filters and security.

Read it

• Report (PDF) and supplement (PDF)
• Sources on GitHub ↗
• Results: main comparison ↗, screened exact search ↗

Version 0.25, released 8 November 2026 (tag paper-v0.21; 0.21 corrects one speed ratio in the conclusion). A technical report, not peer reviewed. Text and figures under CC BY 4.0, code under Apache-2.0.