Install and query MorletDB

MorletDB is a developer-alpha PostgreSQL extension; the current release is 0.3.14-alpha.1. The SQL extension is named pg_morlet. Run these examples in a new disposable database; application roles should receive only the grants they need.

Fastest start: the release image

Each release publishes a prebuilt image, ghcr.io/merkhet-io/morletdb:<version>: PostgreSQL 17.6 (Debian bookworm) with pg_morlet installed, for Linux amd64 and arm64, with an SBOM per architecture and build provenance (releases). Nothing is compiled. Choose your own password; the port is bound to localhost and the data is kept in the named volume morletdb-data.

docker run -d --name morletdb -e POSTGRES_USER=morlet -e POSTGRES_PASSWORD=change-me \
  -e POSTGRES_DB=morlet -p 127.0.0.1:54329:5432 -v morletdb-data:/var/lib/postgresql/data \
  ghcr.io/merkhet-io/morletdb:0.3.14-alpha.1
docker exec morletdb psql -U morlet -d morlet -c "CREATE EXTENSION pg_morlet"

The image does not include pgvector, so probe search needs a source installation or an image you build on top of it; exact and screened search work as they are. A newer image does not upgrade the SQL objects of an existing database: run ALTER EXTENSION pg_morlet UPDATE after switching images. The update to 0.3.8-alpha.1 (SQL 0.3.6) also pins the sample and derived columns of existing collections to pglz TOAST compression; it changes the catalog only and does not rewrite stored values.

Install on an existing server: Debian packages

From 0.3.5-alpha.1, each release attaches Debian packages for servers that run the PostgreSQL project’s packages (apt.postgresql.org): postgresql-17-pg-morlet and postgresql-18-pg-morlet, for amd64 and arm64, built on Debian bookworm. Before release, each one is installed with apt into a clean official postgres image and runs the quickstart, a restart and the quickstart again. Download the package for your server and SHA256SUMS from the release, then:

sha256sum -c SHA256SUMS --ignore-missing
gh attestation verify postgresql-17-pg-morlet_<version>_<arch>.deb --owner merkhet-io
sudo apt install ./postgresql-17-pg-morlet_<version>_<arch>.deb
psql -d your_database -c "CREATE EXTENSION pg_morlet"

The file names use . where the package version has ~ (0.3.5.alpha.1 for version 0.3.5~alpha.1-1), because GitHub release assets cannot contain ~. The packages are trialed only on Debian bookworm images; other distributions are not tested. pgvector, for probe search, is a separate package, for example postgresql-17-pgvector. Upgrade an existing database with ALTER EXTENSION pg_morlet UPDATE after installing a newer package. Packages for PostgreSQL 19 follow 19.0 (releases).

On PostgreSQL 18 and later, pg_morlet can also be loaded from a directory outside the server, such as an image volume in a container, through extension_control_path and dynamic_library_path (from release 0.3.9-alpha.1, SQL 0.3.7, with a ready-made extension-only image ghcr.io/merkhet-io/pg_morlet:0.3.12-alpha.1-pg18; how).

Docker walkthrough with synthetic data

Install Git and Docker with Compose v2, then run:

git clone https://github.com/merkhet-io/morletdb.git
cd morletdb
docker compose run --rm walkthrough

The first run builds the extension image and installs Python dependencies. It imports synthetic recordings, searches a known interval and reads matches back. Internet access is required. The synthetic data is a functionality demonstration, not evidence of diagnostic accuracy. The default walkthrough does not install pgvector.

To open SQL in the running demo database:

docker compose exec db psql -U morlet -d morlet

The demo listens on localhost port 54329, database/user morlet, password morlet-demo-only. These are public local-demo credentials. docker compose down preserves the named volume; adding -v deletes its data.

Store, search, read back

The following example creates its own collection, independent of the Docker walkthrough. Run it once in a fresh database with pg_morlet installed. It stores two synthetic recordings as raw-only inputs, then builds envelopes explicitly. This is the envelope-search path; it does not create Morlet feature segments.

SELECT morlet_create_stream_collection('public', 'getting_started');

SELECT morlet_start_recording(
  'public.getting_started', 'run-a', 4096, 'vibration', 'g',
  '{"asset":"spindle-1"}'::jsonb, 512, 256, 'zscore', 'none'
);
SELECT morlet_start_recording(
  'public.getting_started', 'run-b', 4096, 'vibration', 'g',
  '{"asset":"spindle-1"}'::jsonb, 512, 256, 'zscore', 'none'
);

SELECT morlet_append_chunk(
  'public.getting_started', 'run-a', 0,
  ARRAY(SELECT sin(i::float8 / 8) * (1 + 0.4 * sin(i::float8 / 300))
        FROM generate_series(0, 8191) AS i ORDER BY i)
);
SELECT morlet_append_chunk(
  'public.getting_started', 'run-b', 0,
  ARRAY(SELECT sin(i::float8 / 8) * (1 + 0.4 * sin(i::float8 / 300))
        FROM generate_series(0, 8191) AS i ORDER BY i)
);

SELECT morlet_enable_envelopes('public.getting_started');
SELECT morlet_build_envelopes('public.getting_started', 'run-a', 4096, 2048);
SELECT morlet_build_envelopes('public.getting_started', 'run-b', 4096, 2048);

SELECT * FROM morlet_search_envelope_interval(
  'public.getting_started', 'run-a', 0, 4096,
  '{"asset":"spindle-1"}'::jsonb, 10
);

SELECT * FROM morlet_read_interval(
  'public.getting_started', 'run-b', 0, 4096
);
SELECT * FROM morlet_stream_status('public.getting_started', 'run-a');
SELECT * FROM morlet_collection_status('public.getting_started');

There are fewer than ten admissible, non-overlapping intervals here; fewer than ten results is expected. Sample coordinates are zero-based, half-open: [0,4096) contains 4096 samples. Source exclusion removes overlapping intervals in the source recording, not every interval of that recording. Smaller distances indicate stronger similarity under the chosen scorer; distances are not probabilities.

Which search to use

MorletDB has two ways to find similar intervals. Exact search compares the query with every eligible stored window and returns the guaranteed nearest ones. Probe search is indexed vector search: pgvector's HNSW index proposes candidates for 13 time-shifted copies of the query, and those candidates are re-ranked with the exact distance. Returned distances are exact either way; probe search may miss a true match the index did not propose.

Call morlet_search_similar unless you have a reason not to. It uses probe search when probes are enabled for the window size and exact search otherwise; probe search itself switches to exact search when at most 5,000 windows are eligible, for example under a narrow metadata filter.

Probe search found 99.0–99.8% of exact search's top 10 in E7 (recall@10 0.990–0.998). If missing about one match in a hundred is acceptable, for exploration or a quick “have we seen this before?”, it is the faster choice on large archives; if not, use exact search, with a metadata filter where you can.

Exact search runs as one parallel statement: raise max_parallel_workers_per_gather and, for large envelope tables, the table's parallel_workers storage parameter. It fails above max_candidates eligible windows rather than searching a subset. SET morlet.exact_screen = off disables the screen (SQL 0.3.3) and scores every window, with identical results.

On small machines, exact search uses its prepared query matrices only when they fit a quarter of the CPU’s last-level cache and otherwise scores windows directly, with bitwise the same distances (from 0.3.8-alpha.1). On a Raspberry Pi 3B+, an N1 query over 1,341 windows of 16,384 samples takes about 1.5 s instead of 15.1 s; desktops are unchanged. MORLET_SCORE_CACHE_BYTES in the server’s environment overrides the cache budget (edge latency ↗).

Optional indexed probe search

Install pgvector server files separately for the same PostgreSQL installation, then enable it in the same database. The commands below continue the example above.

CREATE EXTENSION IF NOT EXISTS vector;
SELECT morlet_enable_envelope_probes('public.getting_started', 4096);
SELECT * FROM morlet_search_envelope_probes(
  'public.getting_started', 'run-a', 0, 4096,
  '{"asset":"spindle-1"}'::jsonb, 10
);

The function routes small eligible galleries (at most 5,000 windows by default) to exact search, so this tiny example exercises that fallback. On larger galleries it unions shifted HNSW candidates and re-ranks them with the envelope scorer: candidate selection is approximate even though returned distances are recomputed exactly on stored envelopes. It may return fewer than top_k after exclusions because the per-probe budget is fixed.

Envelopes are opt-in and are not automatically built by append. Call the bounded, resumable builder until the intended windows are covered. Once probes are enabled, the builder writes corresponding vector rows in the same transaction. To publish appended samples and new representations together, explicitly put append and build calls in one transaction. Under READ COMMITTED, the multi-statement probe function does not promise one atomic search snapshot.

Storage codecs

Every stored sample is compressed losslessly by Grizzbit, MorletDB’s chunk codec: a value read back has the same bits as the value written, and with morlet.verify_appends on (the default) every chunk is decoded again and compared before it is stored. You do not need to choose anything; the codec picks the smallest coding per chunk and records it in the chunk’s first byte.

To use Grizzbit 2.1, set it for a database (or a session, or the whole server). Only chunks written afterwards use version 5; existing chunks keep their format, and a collection may mix both:

ALTER DATABASE plant SET morlet.chunk_format = 'grizzbit21';

• Worth it for waveforms on a value grid. It gains little on plain counts (ECG, seismic) and nothing on floats without a grid, and it encodes at about half to three quarters of Grizzbit 1’s speed.
• Compatibility: readers older than the release that ships it cannot read version-5 chunks; compressed transfer gives such clients plain samples. Embedded-edition files always use Grizzbit 1.
• Turning it off: ALTER DATABASE plant RESET morlet.chunk_format; Chunks already written stay readable.

How to check which format a recording uses, every option and the full results: Grizzbit ↗ and the format specification ↗.

Python SDK

The morletdb Python package (sdk/python, alpha, not yet on PyPI) lets you ingest your own waveform, read exact intervals, search for similar intervals and inspect the results in an afternoon. It needs only numpy and psycopg 3.2 or later, works with pg_morlet SQL 0.3.2 and later, including the release image (newer methods name the SQL version they need), and adds no database features of its own: every call is one of the SQL functions in the reference below. Releases attach wheels with the native decoder for compressed transfer (Linux x86-64 and arm64, macOS 11+ on arm64) and a pure wheel for other platforms (install a wheel); from a checkout, pip install ./sdk/python.

First run

docker run -d --name morletdb -e POSTGRES_USER=morlet -e POSTGRES_PASSWORD=change-me \
  -e POSTGRES_DB=morlet -p 127.0.0.1:54329:5432 ghcr.io/merkhet-io/morletdb:0.3.14-alpha.1
docker exec morletdb psql -U morlet -d morlet -c "CREATE EXTENSION IF NOT EXISTS pg_morlet"
pip install "./sdk/python[plot]"          # from a checkout of the repository
DATABASE_URL=postgresql://morlet:change-me@127.0.0.1:54329/morlet python examples/python/quickstart.py

If the container from Fastest start is already running, skip the first two commands. The Docker walkthrough’s database also uses port 54329, so stop it first (docker compose down) or change the port.

The example generates two recordings of two minutes each at 8 kHz with impacts and electrical transients at known times, reads 0.5 s of an impact by time, searches from it and plots the query with its best matches. The four impacts rank first (distance about 0.01), ahead of the electrical transients (about 0.16–0.19).

Use

from datetime import datetime, timezone
import numpy as np
import morletdb

samples = np.load("pump7_accel.npy")                     # 1-D float or integer array
with morletdb.Client.connect("postgresql://...") as db:  # commits on a clean exit
    db.create_collection("public.signals")
    rec = db.ingest("public.signals", "pump-7/accel-x", samples, sample_rate_hz=25_600,
                    channel="accel-x", unit="m/s^2", metadata={"asset": "pump-7"},
                    start_time=datetime(2026, 3, 1, 8, 0, tzinfo=timezone.utc),
                    search_stride=512)                    # windows every 512 samples: see below
    iv = db.read("public.signals", "pump-7/accel-x", 40_960, 45_056)     # samples [40960, 45056)
    result = db.search("public.signals", "pump-7/accel-x", 40_960, metadata_filter={"asset": "pump-7"})
    for m in result.matches:
        print(m.rank, m.distance, m.ref)                 # recording + half-open sample range
        original = db.read_ref(m.ref).samples            # the stored samples, bit for bit

What the SDK guarantees

• Lossless samples. Values are stored as float64 and read back bit for bit; the SDK never resamples, normalizes or lowers precision. Search uses a derived envelope representation only to find candidates; every result is read from the original samples.
• Explicit coordinates. Intervals are half-open sample ranges [start, end). start_time is the UTC time of sample 0, a timezone-aware datetime or integer nanoseconds (naive datetimes are refused); sample n is at start + n / sample_rate_hz.
• Distance, not probability. The distance is one minus the best correlation of the two envelopes over small time shifts: about 0 for the same shape, smaller is more similar, comparable only within one window size.
• Where search windows start. Stored windows of 4,096 samples start every search_stride samples (default 2,048), and search tolerates shifts of about ±300 samples. Events between window starts can be missed with the default; search_stride=512 covers every offset at four times the envelope storage. In the example, the default found one of four impacts and 512 found all four.
• Your transactions. Methods never commit on a connection you pass in. Identical re-ingests are no-ops; conflicting ones raise and leave the stored samples unchanged.

Full contract, errors and limitations: Python SDK guide ↗ · example: quickstart.py ↗.

Connect an AI agent (MCP)

tools/mcp/morlet_mcp.py is a read-only Model Context Protocol server. It lets an agent such as Claude work through a recording archive the way an engineer would: see what is recorded, characterize an interval, find where similar behavior happened, read the original samples of every candidate and cite them.

Install the dependencies, then register the server with your client:

python -m pip install -r tools/mcp/requirements.txt

# Claude Code
claude mcp add morletdb --env DATABASE_URL=postgresql://signals_reader@localhost/signals \
  -- python /path/to/morletdb/tools/mcp/morlet_mcp.py

For Claude Desktop, add the server to claude_desktop_config.json:

{"mcpServers": {"morletdb": {
  "command": "python",
  "args": ["/path/to/morletdb/tools/mcp/morlet_mcp.py"],
  "env": {"DATABASE_URL": "postgresql://signals_reader@localhost/signals"}}}}

Connect as a dedicated read-only role:

CREATE ROLE signals_reader LOGIN PASSWORD '...';
GRANT USAGE ON SCHEMA public TO signals_reader;
GRANT SELECT ON ALL TABLES IN SCHEMA public TO signals_reader;  -- or only the collection's tables
-- tables created later (new collections) by the role that runs this:
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT ON TABLES TO signals_reader;

Safety. Every call runs in a read-only transaction with a statement timeout (MORLET_MCP_TIMEOUT_MS, default 30 s), as the role in DATABASE_URL, so privileges and row-level security decide what the agent sees. Errors such as an unknown collection or an interval of the wrong length come back to the agent as tool errors it can correct. Importing, labeling and building search structures stay with the operator.

A worked investigation. On the seismic development data of the technical report (four CX stations, 1–14 March 2025), an agent asked to find the M2.6 earthquake offshore Tarapacá on 14 March at station PB01 and to search for similar signals located the moment with locate_time, described it and searched. Three of the five closest intervals across four stations and two weeks were other cataloged earthquakes (M2.7–3.0); the other two had no catalog entry. For a weak signal, such as a distant earthquake only a few times above the background, compare_intervals against the background around it shows the frequency content of what is new rather than of the background. Where a search window starts changes which hits rank first, so when an agent knows when something happened, search_from_time finds the onset and frames the window itself. The full transcript, with tool calls and results, is in the MCP guide ↗.

Search intervals must have the stored window length (4,096 samples unless built otherwise; list_collections shows it). From SQL 0.3.2, search_similar uses probe search where it is enabled and exact search otherwise. No data yet? The agent demo ↗ sets up four days of public seismic recordings and the server in Docker, with a second, offline MCP server for station coordinates, an earthquake catalog snapshot and predicted arrival times. The machine demo ↗ sets up a bearing run to failure (NASA/IMS). Or import WAV, EDF, miniSEED or CSV files ↗ first. Check the server with DATABASE_URL=... python tools/mcp/smoke_stdio.py.

Run it unattended: the steward

tools/steward (from 0.3.8-alpha.1) keeps a MorletDB database healthy on an edge machine without a DBA. It runs the rebuildable upkeep (envelopes, chunk summaries, roll-ups, context values, a rolling morlet_verify pass every 7 days and ANALYZE of changed tables) in one budgeted loop, only below a load threshold (--load-per-cpu, default 0.7) and within the disk ladder, with a circuit breaker per job. It records an action row for every call and keeps one plain status, the view morlet_steward.status: one line per item with cause, consequence, action and who, for people and HMIs. Its role can build derived data and summaries and nothing that removes stored samples; its database test shows that DELETE, UPDATE of samples, TRUNCATE and DROP are refused.

pip install -r tools/steward/requirements.txt
# once, as the database owner: schema, roles (morlet_steward, morlet_approver), grants
python tools/steward/steward.py init  --dsn "postgresql://owner@host/db"
python tools/steward/steward.py grant --dsn "postgresql://owner@host/db"   # again after new collections
psql -c "ALTER ROLE morlet_steward LOGIN PASSWORD '...'"
# the service
python tools/steward/steward.py run --dsn "postgresql://morlet_steward@host/db" --data-path /var/lib/postgresql/data
# for people and health checks
python tools/steward/steward.py status --dsn ...       # NORMAL / ATTENTION / ACTION REQUIRED, one action per item
python tools/steward/steward.py check --dsn ...        # exit 0, 1 or 2
python tools/steward/steward.py plans --dsn ...        # proposed plans and their state
python tools/steward/steward.py approve 12 --dsn ...   # another person (morlet_approver) approves
python tools/steward/steward.py apply 12 --dsn ...     # and applies it, in one transaction

Disk and the ingest hold. The steward watches free space as time remaining (attention at 14 days, pause at 3). At the reserve (by default the larger of 5% and 2 GiB; --reserve-fraction, --reserve-bytes) it sets the ingest hold with morlet_set_ingest_hold: appends that would store new samples then fail with SQLSTATE 53100 and the reason, before PostgreSQL itself runs out of disk, while identical replays still pass. When space is free again it releases the hold it set (morlet_release_ingest_hold); a hold set by a person stays until a person releases it. Writers can check beforehand with morlet_check_ingest_hold(), which raises the same error while a hold is set. --data-path is the directory whose filesystem holds the database; if the steward cannot read free space, it pauses all work and says so.

Plans. Anything that removes or changes stored samples, or updates the extension, is a plan. The steward proposes extension updates itself when the server has a newer pg_morlet; a person proposes memory-level changes from a policy dry run (propose-levels COLLECTION MOVES.json). Another person with morlet_approver approves it, and a person applies it with their own privileges, in one transaction. The rules are enforced in SQL (morlet_steward.decide, morlet_steward.apply): no self-approval, the payload tied to the approved hash, expiry after 24 hours, applied once. The steward’s role can propose and read plans, but cannot decide, apply or delete them.

Deploy. deploy/steward/Dockerfile is a sidecar image whose health check is healthy while the status is normal or attention. deploy/steward/compose.yaml runs a release image (MORLET_IMAGE) with data checksums and small-disk WAL settings, a one-shot init and grant, and the steward with the data volume mounted read-only. deploy/steward/morlet-steward.service is a systemd unit for package installs.

Tested. Filling a real 1.2 GB disk on a Raspberry Pi 3B+, the steward set the hold at 295 MB free, appends were refused with SQLSTATE 53100 and never with a PostgreSQL write error, freeing 200 MB resumed ingest 14.5 s later without a person, and every acknowledged sample read back bitwise (disk fill results ↗).

Not yet: dropping derived data under disk pressure by policy, automatic retention proposals, a multi-week soak on the Raspberry Pi, power-cut tests, and the status as a read-only MCP tool. Metrics: morlet-steward metrics serves Prometheus metrics on /metrics, read-only (#197). Design and rules: steward design ↗ · steward README ↗.

Main functions and query primitives

Filters use JSONB containment, for example '{"asset":"spindle-1"}'::jsonb. Eligibility also requires compatible rate, channel, unit and representation/window settings. Exact searches have explicit candidate budgets and fail when exceeded rather than silently searching a prefix. Returned intervals are ordered by distance, recording ID and start sample, with source exclusion and greedy overlap suppression enabled by default on stored-interval searches.

Do not directly edit managed companion tables. Versioned extractors, original samples and representations have different roles. This guide uses the active continuous-recording API, not the legacy bounded-row APIs. Functions not listed here (relation lookups, the chunk codec, checkpoints and envelope internals) are internal helpers and may change between versions. Context, comparison and acquisition workflows are described in the context workflow guide ↗.

Supported versions

This describes the current source and CI configuration, not a promise that every CI run passes. The paper benchmarks use PostgreSQL 17; compatibility tests and performance measurements are separate.

Build a development installation

Install Rust through rustup. On Debian/Ubuntu, install the build prerequisites below. Run pgrx as a non-root OS user; only the apt command needs administrator privileges.

sudo apt-get install build-essential bison flex libreadline-dev zlib1g-dev libclang-dev clang pkg-config libssl-dev
git clone https://github.com/merkhet-io/morletdb.git
cd morletdb
rustup show
cargo install cargo-pgrx --version 0.19.3 --locked
cargo pgrx init --pg17 download
cd extensions/pg_morlet
cargo pgrx run pg17

This builds a user-owned PostgreSQL 17 installation and opens psql. In the selected database:

CREATE EXTENSION IF NOT EXISTS pg_morlet;
SELECT morlet_version();

CREATE EXTENSION enables already-installed extension files; it does not download or compile the extension. No shared_preload_libraries entry is required. Hosting providers must permit installation of this extension. For packaging against an existing PostgreSQL 17 installation, use cargo pgrx package --pg-config /absolute/path/to/pg_config; installation into server directories requires suitable filesystem privileges.

Other PostgreSQL versions build the same way, one Cargo feature at a time, in extensions/pg_morlet:

# PostgreSQL 18
cargo pgrx init --pg18 download
cargo pgrx run pg18 --no-default-features --features pg18
# PostgreSQL 19 (release candidate 19rc1, built from the PostgreSQL source tarball)
cargo pgrx init --pg19 /path/to/19rc1/bin/pg_config
cargo pgrx run pg19 --no-default-features --features pg19

Register a 19rc1 build by its pg_config, since cargo pgrx init --pg19 download fetches 19beta4. Probe search on 18 needs pgvector 0.8.1 or later (CI uses 0.8.7). PostgreSQL 16 and older are not supported. See the compatibility matrix.

Further reading

• AI agents over MCP ↗
• Python SDK guide ↗
• Import recordings from files ↗
• Detailed search contract ↗
• Durable ingestion and envelope lifecycle ↗
• Access control ↗
• Context and compare-to-history ↗
• Operations and recovery ↗
• Operator runbooks ↗
• Steward: usage ↗ and design ↗
• Episodic memory and graded compaction ↗
• PostgreSQL compatibility ↗
• Verification of the chunk codec ↗
• Versioned releases ↗

Function names and signatures on this page follow extensions/pg_morlet/sql/streams.sql of the release; installation commands follow the Cargo features, the release workflow and the release notes. CI runs the Python quickstart against PostgreSQL 17 on every change; the SQL snippets on this page are not run by CI.