pgpipe Changelog

All notable changes to pgpipe are documented here.

The format is based on Keep a Changelog and pgpipe adheres to Semantic Versioning.

This file is the canonical source of truth. It ships inside every .deb / .rpm at /usr/share/doc/pgpipe/CHANGELOG.md and is rendered at https://www.pghorizon.com/pgpipe/changelog/.

“Unreleased” lists changes that are merged but have not yet shipped in a numbered release.

Unreleased

4.0.0 - 2026-07-20

Fixed

• Operator-approved ADD COLUMN ... NOT NULL now freezes the exact target, canonical source proof, literal default, batch size, row cap, and end-to-end deadline before approval. Stale pipeline writes cannot erase authorization, failed attempts remain failed and require explicit re-approval, and applying workflows cannot be resolved concurrently. Partition/inheritance batching is relation-qualified, every inherited/partition relation is locked, enabled destination UPDATE triggers/rules fail closed, domain/custom types and explicit collations remain manual, and the deadline covers connection/preflight, fencing, dependent DML, verification, and commit. The API/dashboard show the reviewed proof and expose its configuration.
• Release version derivation now normalizes vX.Y.Z Git tags, rejects dirty or non-tagged release builds (including untracked files), and verifies the embedded version in local, Docker, and native-package artifacts instead of silently shipping 1.0.0 or dev.
• Failure paths now have enforceable limits: destination connections apply statement_timeout, the retry deadline includes each database attempt, and DLQ writes atomically enforce max_entries without ever evicting unresolved events. Startup reconciles a lowered DLQ cap before source activity, retry attempts have finite deadlines, and concurrent stale workers cannot recreate reclaimed records.
• Dashboard configuration saves no longer mutate the live pipeline while reporting restart_required; pending values remain visible for subsequent edits and take effect only after restart.
• Transient source WAL-stream disconnects now recover in-process with bounded, shutdown-interruptible backoff. Every attempt reloads the durable checkpoint, reconnects the replication transport, and verifies slot/WAL safety before resuming; slot loss and recycled WAL still fail closed.
• Table-level DDL quarantine recovery now treats every retained queue as authoritative: disabling admission or selecting a fresh snapshot cannot bypass historical events, destination apply-ledger cleanup waits until replay completes, later unsupported DDL keeps only its own table blocked, and queue startup/replay is payload-paged. Deterministic-ID conflicts, transform-plan drift, replica-identity index changes, dotted-schema foreign keys, and column-sensitive transform/DDL combinations fail closed. Backend migrations, counters, advisory-lock waits, and duplicate lifecycle metrics are bounded and regression-tested.
• The configuration API/dashboard now expose the complete structured-DDL quarantine subtree. Blocked DDL has an accessible reviewed-resolution dialog, authenticated audit identity comes from the JWT, retained totals are visible without hover, and the dashboard hero shows a degraded DDL-quarantine state.

Added

• Structured DDL now coordinates same-schema table and column renames without parsing or rewriting source SQL. Event-trigger capture records old/new names with stable relation OID/attribute identities and the complete post-rename table shape; planning rejects collisions, schema moves, incomplete metadata, and ambiguous identities. Source selectors, transform keys, mapped aliases, primary/replica-identity writer keys, and tracked metadata advance in WAL order. Multiple renames and dependent DML commit atomically, rollback keeps writer metadata unchanged, and startup reconciles source capture plus durable ledger history so exact table selections resume after rename. The default durable mode also resolves the crash window after destination COMMIT but before state/checkpoint persistence without recreating the old table; best_effort fails closed when that commit state is ambiguous.
• Unsupported source DDL can now use an opt-in, durable table-level quarantine when pgpipe proves the committed transaction is isolated to one independent replicated table. The complete transaction is persisted before checkpoint advancement, later table transactions remain ordered behind the barrier, and unrelated tables continue. Mixed-table, foreign-key, partition, identity-changing, ambiguous, malformed, and over-capacity cases fail closed. BoltDB, SQLite, and PostgreSQL atomically enforce transaction/event/byte bounds; startup restores or replays queues before source streaming, while status, table history, the dashboard, and Prometheus expose retained work.
• The production image now runs as dedicated UID/GID 10001, includes a native TLS-aware liveness/readiness command and Docker health check, and supports a read-only root filesystem with one documented persistent writable state path. The Compose demo continuously exercises dropped capabilities, no-new-privileges, read-only config, and read-only-root operation.
• The container config directory is root-owned and runtime-immutable, while a fresh persistent state volume remains writable by UID/GID 10001. Built-in liveness also handles intentionally headless pipelines without marking the running container unhealthy.
• Tagged releases now publish signed linux/amd64 and linux/arm64 GHCR images plus signed binaries/native packages. Release artifacts include SPDX SBOMs and GitHub provenance; container indexes carry BuildKit SBOM and maximum-mode provenance attestations. Pinned govulncheck/Trivy scanning, dependency review, immutable GitHub Action/base-image references, and weekly Dependabot updates enforce the supply-chain contract.
• Release images are now built once per platform, scanned and signed by exact digest, combined into a verified candidate index, and only then promoted to final tags. Release guards reject off-main, moved, duplicate, and stale-license releases before registry mutation.
• The minimum Go toolchain is now 1.26.5 and pgx is updated to 5.9.2, closing the reachable standard-library TLS/X.509/HTTP findings and pgx placeholder confusion reported by govulncheck when the security gate was introduced.
• Structured DDL replay now safely creates new ordinary primary-keyed tables discovered through PostgreSQL 15+ schema wildcards. The event trigger stores the canonical table shape in the WAL-visible DDL row; the planner accepts only published, non-partitioned, built-in-type tables without defaults, generated/identity columns, custom collations, non-plain primary keys, or non-primary constraints. Exact includes override wildcard exclusions, live selector updates are immutable, dynamic mapped names cannot collide with tracked/reserved tables, and DDL-only or mixed transactions apply atomically behind the destination fence. Every wider CREATE TABLE shape remains durably blocked.
• Pipeline startup now acquires exclusive ownership before any source activity. PostgreSQL state uses a dedicated session advisory lock with a renewed lease; BoltDB and SQLite use an OS sidecar lock. Every takeover advances a stable pipeline fencing generation, checkpoints validate it atomically, and destination DML, COPY, snapshot reset, sequence, heartbeat, and supported DDL mutations hold the destination fence through commit. A second healthy owner is refused, stale writers fail closed, and /api/status plus Prometheus expose owner identity/generation and whether this process currently owns the pipeline. Destination connections are pre-warmed so fence activation immediately follows state takeover; preflight always validates fence privileges, and an ownership marker prevents collisions with application-owned internal names.
• pgpipe verify now verifies row contents rather than counts alone. It streams canonical SHA-256 fingerprints in primary-key order, compares bounded ranges, recursively narrows mismatches, and reports exact missing, extra, and changed identities without exposing non-key row values.
• Verification reports can be persisted to BoltDB, SQLite, or PostgreSQL state with --persist-history. --create-repair-jobs creates deterministic, preview-only repair jobs for mismatched leaf ranges; approval and execution remain explicit operator actions. History and preview jobs commit atomically; detailed identities and generated repair jobs have independent run-wide hard caps while aggregate mismatch counts stay exact.

Changed

• The current configuration schema is version 4 and adds the disabled-by-default source.ddl.quarantine hard-bound subtree. Older supported schema versions continue to load on top of safe defaults.
• Existing config files that explicitly set the newly bounded failure fields to 0 are normalized to finite defaults during runtime loading, preserving upgrade compatibility. New validation and dashboard edits require positive limits.

3.3.0 - 2026-07-04

Fixed

• Safe nullable DDL auto-apply now supports PostgreSQL typmods for common parameterized built-in column types, including varchar(n), char(n), numeric(p,s), and time/timestamp precision, instead of blocking all typmod-bearing ADD COLUMN events.
• DDL ledger APIs now fail closed when the pipeline scope is unavailable and pipeline.name validation rejects explicit blank values. Omitted names still default to pgpipe, but configs that intentionally set pipeline.name to an empty or whitespace-only value must choose a stable non-empty name before startup.

3.2.0 - 2026-06-28

Added

• pgpipe status is now API-first: it prefers the running process’s /api/status view, supports authenticated reads with --api-token or PGPIPE_API_TOKEN, verifies unauthenticated /ready before sending a token, and keeps the previous direct state/source diagnostics as --local plus automatic fallback when the API is unavailable. Explicit API-token failures now fail loudly instead of silently falling back. /api/status also includes the latest persisted checkpoint when available.
• Phase 2 VER-01 read-only verification begins with pgpipe verify. The new command compares configured primary-keyed source tables against their transformed destination table names, reports matched/mismatched/skipped/error table counts, supports --json, bounds query impact with a statement timeout, and exits non-zero on mismatches or table errors by default.
• Phase 2 BACK-01 foundation: live AddTable snapshotting now runs through a coordinated backfill path that captures a diagnostic WAL marker, persists backfill phase/progress, buffers committed streaming WAL and watermarks while the job is active, and replays buffered changes only after the snapshot copy drains so older snapshot rows do not recreate rows deleted during the backfill. Restarts that find an incomplete live backfill now fail closed with an operator-facing recovery message instead of marking the table healthy.
• Phase 2 SCH-01D schema guard mode: schema_guard.mode now supports observe (default) and block_unsafe. In block_unsafe, pgpipe records schema drift and fails startup or live AddTable before writes begin when drift is classified as backfill-required, incompatible, or destructive.
• Phase 2 SCH-01E DDL apply ledger foundation: state backends now persist a DDL ledger keyed by stable event identity, with status, LSN, XID, canonical metadata, bounded diagnostics, and BoltDB/SQLite/PostgreSQL coverage.
• Phase 2 SCH-01F optional safe-additive schema auto-apply: schema_guard.auto_apply_safe_additive can add missing nullable destination columns with no default/identity/generated expression, recording pending, applied, or failed status in the DDL ledger. All other drift remains observe/block-only.

Fixed

• Unimplemented maintenance command types such as pause, resume, resnapshot_table, the repair maintenance-command wrapper, and slot_recovery are now rejected during submit preflight instead of being persisted as accepted commands that fail later with a generic “not implemented” result. Targeted repair remains available through /api/repair.
• Heartbeat source preflight now matches startup setup for managed heartbeat tables, so pgpipe validates the required source database or schema CREATE readiness before allowing startup-created heartbeat tables.

3.1.0 - 2026-06-24

Security

• Exact session invalidation via a persisted session version. A new pgpipe-auth-version file (mode 0600, beside the state DB) stores the version carried by every dashboard JWT. Password changes, recovery resets, and “sign out all sessions” rotate the version, rejecting every older token without timestamp or same-second race conditions. Missing version files are initialized from a random positive base so recreation cannot revive old tokens; corrupt, unreadable, or loosely permissioned files fail startup. Runtime read errors and version rollbacks preserve the last trusted version instead of resurrecting revoked sessions.
• Effective dashboard credentials are tracked using a keyed HMAC fingerprint. Changing an env/YAML/file-managed username or password outside pgpipe rotates the session version automatically on the next startup. A missing established fingerprint also rotates sessions. Invalid, unreadable, or loosely permissioned fingerprints are treated as an unknown credential change: sessions are invalidated and the fingerprint is repaired before startup continues. The password and fingerprint are never logged.
• Public auth discovery and protected account details are separated: GET /api/auth/mode is public and returns only whether auth is enabled; GET /api/auth/me requires a valid JWT and returns account/security capabilities.

Added

• TST-01 required CI: GitHub Actions now runs separate fast and integration jobs. Fast checks cover formatting, vet, build, shipped-config validation, short unit tests with coverage, race tests, and license-bundle drift; integration checks run Docker-backed PostgreSQL tests, focused operational scenarios, and the full Docker demo. CI uploads coverage/log artifacts and includes manual failure probes for unit-test, formatting, and integration reporting drills.
• TST-02 hermetic PostgreSQL integration harness: Go integration tests now start isolated source/destination PostgreSQL containers per package when explicit DSNs are not supplied. The harness uses random host ports, deterministic fixtures, unique slot/publication helpers, polling/table comparison helpers, and Docker/PostgreSQL diagnostics captured as artifacts on failure.
• REL-01 safe shutdown: normal termination now marks pgpipe unready, stops source intake, drains accepted streaming work, persists a final safe checkpoint with a fresh bounded context, and then closes. Shutdown progress is exposed through status fields and Prometheus metrics, final-checkpoint failures are counted, and a second SIGINT/SIGTERM forces process exit for stuck graceful shutdowns.
• OPS-02 meaningful readiness: /ready now reports component states for the source, destination, state store, replication slot, and checkpoint progress. Readiness fails on dependency outages, missing/stale/lost slot samples, stale checkpoints with WAL waiting, and graceful shutdown, while healthy idle pipelines remain ready and startup grace labels first-sample gaps without prematurely marking the pipeline ready.
• CFG-01 strict configuration validation: pgpipe.yaml now rejects unknown or misspelled YAML keys, fails fast on unset or malformed ${VAR} references, and recognizes top-level config_version: 1 as the current schema. Missing config_version is treated as version 1 for existing configs; explicit older or future versions fail before replication starts.
• pgpipe status provides a read-only terminal status view for operators: dashboard readiness when enabled, latest saved checkpoint LSN, source replication slot state, WAL lag/retained WAL bytes, DLQ depth, and the latest table-state error when one has been persisted. --json emits the same report for scripts. Local BoltDB/SQLite status reads use read-only open paths, and missing or locked local state is reported explicitly instead of being treated as an empty state store.
• Missing source replication slots are now classified as fatal slot-loss errors with operator recovery guidance. On resume, pgpipe stops before advancing the checkpoint and explains the safe choices: restore the slot or perform an explicit fresh snapshot/re-copy; it does not silently recreate the slot at the current WAL position. Operators who intentionally want a rebuild path can set source.replication.missing_slot_policy: recreate_with_snapshot; pgpipe will only use it together with snapshot.reset_destination: truncate when the slot is missing, invalidated, or cannot replay the saved checkpoint, then recreate the slot by running a fresh snapshot before streaming.
• The streaming checkpoint loop now advances the source replication slot through decoded WAL that produces no destination work. PostgreSQL XLogData records now emit an in-band watermark after their logical messages are decoded and handed to the pipeline; that watermark is sequenced behind any earlier source work before it can be checkpointed. This lets decoded stream positions that produce no destination rows become checkpointable without using keepalive ServerWALEnd as a checkpoint candidate or weakening the “never acknowledge past unapplied work” invariant.
• COR-05 explicit durability contract: destination.write.durability now names the operator-selected guarantee (durable default, best_effort opt-in). Durable mode rejects async_commit, forces synchronous_commit=on for pgpipe-controlled destination transactions, creates a destination pgpipe.apply_ledger, and records row-changing live source transactions in that ledger in the same transaction as row changes so restart can skip already-applied commits after a checkpoint/state-store failure. Documentation now calls out the durable-mode throughput cost: synchronous destination commits plus one apply-ledger row per row-changing live source transaction. The ledger now stores a sortable pg_lsn shadow column, drops a duplicate legacy index, and cleans rows below the durable checkpoint after the configured destination.write.apply_ledger.retention window. Cleanup is scheduled outside the checkpoint/resume critical path, bounded by destination.write.apply_ledger.cleanup_batch_size, and ages rows by destination ledger timestamps so old source WAL does not immediately expire a newly-created replay proof. Existing durable-mode configs inherit the 30-day cleanup default on upgrade; set retention: 0s to keep ledger rows indefinitely.
• pgpipe admin skip-commit --lsn <commit-lsn> --xid <source-xid> --reason <why> --yes records an audited manual whole-transaction skip in the destination apply ledger for durable-mode recovery. The default remains halt-on-bad-transaction; skips are never automatic, require an XID match, apply to the whole committed source transaction, emit a WARN, and increment pgpipe_apply_ledger_manual_skips_total.
• COR-04 transform plan safety: transforms: now compiles into one immutable, versioned plan with strict startup validation. filter, mask, and rename are wired from config, identity columns cannot be removed or masked, and DLQ entries record the transform plan version that produced the stored event. Checkpoint resume now rejects transform-plan drift, live-added tables use the transformed destination shape, final destination.schema_mapping collisions are rejected at startup, and fixed-token masks are limited to text-compatible columns. Upgrade note: pipelines with configured transforms and checkpoints created before transform-plan versioning cannot safely resume in place; run a fresh snapshot/re-copy, or restore the previous pgpipe version/config before resuming.
• Transaction spill controls under destination.write.spill: configurable spill root directory, in-memory event threshold, and per-source-transaction byte cap. The default spill directory is co-located beside the state DB in a per-pipeline subdirectory and is swept on startup after hard crashes.
• Dashboard Security tab — change the admin password and sign out all sessions from the browser, built on the session version above. POST /api/auth/change-password verifies the current password, persists a new one (≥12 chars), and invalidates every other session while keeping the caller signed in via a freshly issued token; POST /api/auth/signout-all invalidates every session including the caller’s. Change-password is offered only when the password is file-managed — env/YAML-managed installs see a read-only note (an in-app change would be silently overwritten at the next restart), surfaced via explicit password_source, password_changeable, and session_invalidation_available fields on protected GET /api/auth/me. The live log stream (SSE) now re-checks its token version on each heartbeat, so an open stream also terminates on sign-out within ~15s.
• pgpipe admin reset-password [--generate | --password <value>] — a host-level recovery command that resets the persisted dashboard password and invalidates every existing session in one step. It refuses when the password is supplied via the PGPIPE_DASHBOARD_PASSWORD env var or server.auth.password (those win at startup; change them at their source) and warns when run as root so the rewritten state files stay readable by the service user. Sessions are invalidated immediately; the new password takes effect on the next restart.

Changed

• pgpipe validate now rejects state.backend: postgres unless state.postgres.dsn is set, matching the runtime requirement used by pgpipe start and pgpipe status.
• Snapshot, streaming, destination initialization, and DLQ retry metadata now use the same compiled transform plan. Filter/mask rules apply consistently to INSERT, UPDATE, DELETE old-row data, and snapshot rows; transform failures stop the pipeline before a safe checkpoint can advance.
• Live streaming now preserves COR-01 source transaction atomicity when DLQ is enabled: committed source transactions are not split into per-row DLQ entries; a destination write failure aborts the pipeline and leaves the checkpoint at the previous safe commit. Snapshot batches still use DLQ isolation where row quarantine is correctness-safe.
• Primary-key-changing UPDATE failures now include targeted guidance for duplicate-key (23505) and foreign-key (23503) restart loops, and the transaction-level quarantine / delete+insert recovery modes are documented as explicit future opt-ins.
• Deploying this auth hardening signs out existing dashboard sessions once because pre-upgrade tokens do not carry a session version.
• File-backed BoltDB/SQLite deployments support exact session invalidation. PostgreSQL-backed state currently reports password change and sign-out-all as unavailable instead of showing controls that would fail; PostgreSQL-backed auth-state persistence remains a future enhancement.

Fixed

• UPDATE replay fallback now refuses to insert a fresh row when unresolved TOAST columns would be omitted, and TOAST cache re-keying evicts the old identity even when the new identity is incomplete.
• Deep-linking to #security while dashboard authentication is disabled now returns to the Dashboard instead of exposing a half-populated Security tab with unusable actions.
• Repairing a loose-permission credential fingerprint intentionally signs out existing sessions even when the fingerprint content would otherwise match; pgpipe treats permission drift as untrusted detector state and self-heals the file back to mode 0600.
• A whitespace-only PGPIPE_DASHBOARD_PASSWORD is now treated as “not set” rather than hard-failing startup. The env tier in LoadOrCreatePassword now gates on the trimmed value, so a stray-whitespace export falls through to the config / persisted-file / first-run path and the service starts. This also closes a classification mismatch with admin reset-password, which already trims: before, that command could rotate the password file and report success on a whitespace-only env var that the next pgpipe start would still reject.

3.0.0 - 2026-06-18

Security

• Exact session invalidation via a persisted session version. A new pgpipe-auth-version file (mode 0600, beside the state DB) stores the version carried by every dashboard JWT. Password changes, recovery resets, and “sign out all sessions” rotate the version, rejecting every older token without timestamp or same-second race conditions. Missing version files are initialized from a random positive base so recreation cannot revive old tokens; corrupt, unreadable, or loosely permissioned files fail startup. Runtime read errors and version rollbacks preserve the last trusted version instead of resurrecting revoked sessions.
• Effective dashboard credentials are tracked using a keyed HMAC fingerprint. Changing an env/YAML/file-managed username or password outside pgpipe rotates the session version automatically on the next startup. A missing established fingerprint also rotates sessions. Invalid, unreadable, or loosely permissioned fingerprints are treated as an unknown credential change: sessions are invalidated and the fingerprint is repaired before startup continues. The password and fingerprint are never logged.
• Public auth discovery and protected account details are separated: GET /api/auth/mode is public and returns only whether auth is enabled; GET /api/auth/me requires a valid JWT and returns account/security capabilities.

Added

• The streaming checkpoint loop now advances the source replication slot through decoded WAL that produces no destination work. PostgreSQL XLogData records now emit an in-band watermark after their logical messages are decoded and handed to the pipeline; that watermark is sequenced behind any earlier source work before it can be checkpointed. This lets decoded stream positions that produce no destination rows become checkpointable without using keepalive ServerWALEnd as a checkpoint candidate or weakening the “never acknowledge past unapplied work” invariant.
• COR-05 explicit durability contract: destination.write.durability now names the operator-selected guarantee (durable default, best_effort opt-in). Durable mode rejects async_commit, forces synchronous_commit=on for pgpipe-controlled destination transactions, creates a destination pgpipe.apply_ledger, and records row-changing live source transactions in that ledger in the same transaction as row changes so restart can skip already-applied commits after a checkpoint/state-store failure. Documentation now calls out the durable-mode throughput cost: synchronous destination commits plus one apply-ledger row per row-changing live source transaction. The ledger now stores a sortable pg_lsn shadow column, drops a duplicate legacy index, and cleans rows below the durable checkpoint after the configured destination.write.apply_ledger.retention window. Cleanup is scheduled outside the checkpoint/resume critical path, bounded by destination.write.apply_ledger.cleanup_batch_size, and ages rows by destination ledger timestamps so old source WAL does not immediately expire a newly-created replay proof. Existing durable-mode configs inherit the 30-day cleanup default on upgrade; set retention: 0s to keep ledger rows indefinitely.
• pgpipe admin skip-commit --lsn <commit-lsn> --xid <source-xid> --reason <why> --yes records an audited manual whole-transaction skip in the destination apply ledger for durable-mode recovery. The default remains halt-on-bad-transaction; skips are never automatic, require an XID match, apply to the whole committed source transaction, emit a WARN, and increment pgpipe_apply_ledger_manual_skips_total.
• COR-04 transform plan safety: transforms: now compiles into one immutable, versioned plan with strict startup validation. filter, mask, and rename are wired from config, identity columns cannot be removed or masked, and DLQ entries record the transform plan version that produced the stored event. Checkpoint resume now rejects transform-plan drift, live-added tables use the transformed destination shape, final destination.schema_mapping collisions are rejected at startup, and fixed-token masks are limited to text-compatible columns. Upgrade note: pipelines with configured transforms and checkpoints created before transform-plan versioning cannot safely resume in place; run a fresh snapshot/re-copy, or restore the previous pgpipe version/config before resuming.
• Transaction spill controls under destination.write.spill: configurable spill root directory, in-memory event threshold, and per-source-transaction byte cap. The default spill directory is co-located beside the state DB in a per-pipeline subdirectory and is swept on startup after hard crashes.
• Dashboard Security tab — change the admin password and sign out all sessions from the browser, built on the session version above. POST /api/auth/change-password verifies the current password, persists a new one (≥12 chars), and invalidates every other session while keeping the caller signed in via a freshly issued token; POST /api/auth/signout-all invalidates every session including the caller’s. Change-password is offered only when the password is file-managed — env/YAML-managed installs see a read-only note (an in-app change would be silently overwritten at the next restart), surfaced via explicit password_source, password_changeable, and session_invalidation_available fields on protected GET /api/auth/me. The live log stream (SSE) now re-checks its token version on each heartbeat, so an open stream also terminates on sign-out within ~15s.
• pgpipe admin reset-password [--generate | --password <value>] — a host-level recovery command that resets the persisted dashboard password and invalidates every existing session in one step. It refuses when the password is supplied via the PGPIPE_DASHBOARD_PASSWORD env var or server.auth.password (those win at startup; change them at their source) and warns when run as root so the rewritten state files stay readable by the service user. Sessions are invalidated immediately; the new password takes effect on the next restart.

Changed

• Snapshot, streaming, destination initialization, and DLQ retry metadata now use the same compiled transform plan. Filter/mask rules apply consistently to INSERT, UPDATE, DELETE old-row data, and snapshot rows; transform failures stop the pipeline before a safe checkpoint can advance.
• Live streaming now preserves COR-01 source transaction atomicity when DLQ is enabled: committed source transactions are not split into per-row DLQ entries; a destination write failure aborts the pipeline and leaves the checkpoint at the previous safe commit. Snapshot batches still use DLQ isolation where row quarantine is correctness-safe.
• Primary-key-changing UPDATE failures now include targeted guidance for duplicate-key (23505) and foreign-key (23503) restart loops, and the transaction-level quarantine / delete+insert recovery modes are documented as explicit future opt-ins.
• Deploying this auth hardening signs out existing dashboard sessions once because pre-upgrade tokens do not carry a session version.
• File-backed BoltDB/SQLite deployments support exact session invalidation. PostgreSQL-backed state currently reports password change and sign-out-all as unavailable instead of showing controls that would fail; PostgreSQL-backed auth-state persistence remains a future enhancement.

Fixed

• UPDATE replay fallback now refuses to insert a fresh row when unresolved TOAST columns would be omitted, and TOAST cache re-keying evicts the old identity even when the new identity is incomplete.
• Deep-linking to #security while dashboard authentication is disabled now returns to the Dashboard instead of exposing a half-populated Security tab with unusable actions.
• Repairing a loose-permission credential fingerprint intentionally signs out existing sessions even when the fingerprint content would otherwise match; pgpipe treats permission drift as untrusted detector state and self-heals the file back to mode 0600.
• A whitespace-only PGPIPE_DASHBOARD_PASSWORD is now treated as “not set” rather than hard-failing startup. The env tier in LoadOrCreatePassword now gates on the trimmed value, so a stray-whitespace export falls through to the config / persisted-file / first-run path and the service starts. This also closes a classification mismatch with admin reset-password, which already trims: before, that command could rotate the password file and report success on a whitespace-only env var that the next pgpipe start would still reject.

2.1.0 - 2026-06-13

A dashboard release: a top-to-bottom overhaul of the built-in web UI — visual system, information architecture, DLQ management, accessibility, and a sweep of honesty fixes so the dashboard never claims more than the engine actually does. No configuration, CLI, or wire changes; the replication engine is unchanged from v2.0.0, so upgrading is a drop-in binary/package replacement.

Added

• DLQ management on the dashboard: the unresolved-entry count is surfaced on the Errors card and as a nav badge, with a Discard-all drain action, per-entry copy buttons, and relative timestamps. “Mark Resolved” is renamed to Discard to match what it actually does.
• A status hero with a dedicated Throughput card and honest sparklines, plus a detail strip showing replication-slot liveness, a copyable LSN, and checkpoint age.
• Hash-based tab routing: the active tab lives in the URL, so refresh, the browser Back button, and deep links to a specific tab (e.g. #errors) all work, and screen readers get aria-current on the active nav item.
• A responsive breakpoint that collapses the sidebar to an icon rail at phone width, plus a keyboard-operability baseline across the dashboard.
• Logs tab: a catch-up pill, an honest pause state, a debounced filter, and a download button.
• Setup wizard: an unload guard, recovery from an expired token or a gone server, table search, copy-from-source, and a finish line that shows the saved config path and the exact start command.
• Config editor: tab ergonomics (a default-open section and expand-all), a persistent restart-required banner after a save, busy states on every async action button, and read-only fields that explain why they can’t be edited.
• The real build version is shown in the UI instead of a hardcoded string.

Changed

• A complete visual refresh built on semantic design tokens (color, type, spacing, mono), with inline styles swept into reusable components, pill radii standardized, emoji iconography replaced by inline SVGs (plus an SVG favicon), and native dark browser chrome via color-scheme / theme-color.
• Reworked navigation information architecture with a distinct identity per view; bespoke buttons folded into one .btn system; a skeleton first paint.
• htmx is now vendored into the binary and served from 'self'; the Content-Security-Policy is tightened to drop the unpkg CDN origin.
• The setup wizard is de-forked from its own stylesheet onto the shared theme.css design system.

Fixed

• Polling refreshes no longer destroy in-progress user state, and session expiry preserves mid-edit work and explains itself; stale content degrades visually and Retry is non-destructive.
• The login page no longer consumes login rate-limit budget on page load.
• Honesty: removed the DDL Detection section from the config editor (DDL replication is not implemented in this build), and corrected three false capability claims in the embedded Docs tab.
• Accessibility: ARIA live regions, proper input labels, a global reduced-motion respect, --text-muted raised to WCAG AA contrast, and tabular numerals on live values.
• The pre-auth flash of dashboard chrome is gated, and the user row is hidden when there is no session.
• Tables: the add-table form is clarified (labels, what “snapshot” means, Enter to submit), and per-table read counts are labeled “since start” so a restart reading zero is no longer mistaken for a stall.
• Lag and uptime now format with hour/day branches; severity thresholds are explained; a fully-caught-up pipeline is colored green; the initializing phase is styled, with a neutral fallback for unknown states.
• Error toasts persist until dismissed (hover pauses auto-dismiss); first-contact empty states for Tables and the DLQ; invalid duration edits are rejected instead of silently dropped; and the dead-end “Retry Save” button is replaced with “Back to editing”.

2.0.0 - 2026-06-10

Security

• BREAKING: /metrics is now closed by default. The endpoint honours metrics.{enabled,path,public,auth_token}; a default install requires the auto-generated bearer token printed on first run, instead of exposing metrics openly. Existing Prometheus scrapers must send the bearer token, or set metrics.public: true to restore the old open behaviour. This default change is the reason this release is a new major version.

Fixed

• Destination upsert-key correctness. A chain of fixes closes the SQLSTATE 42P10 (“no unique or exclusion constraint matching the ON CONFLICT specification”) class that previously surfaced per-row at write time (DLQ-quarantined or pipeline abort) instead of as a clear startup error:
  • Name the ON CONFLICT arbiter from the real primary key rather than the replica-identity keys.
  • Correct REPLICA IDENTITY FULL upserts and make destructive snapshots crash-safe.
  • Validate the destination upsert key at startup and mirror USING INDEX unique constraints, failing loudly with a copy-pasteable ALTER TABLE ... ADD CONSTRAINT ... UNIQUE fix.
  • Exclude DEFERRABLE constraints from the arbiter check — PostgreSQL rejects them as ON CONFLICT arbiters at write time (SQLSTATE 55000).
  • Rewrite the DLQ 42P10 quarantine hint to point at the destination constraint. It previously told operators to check that the source’s REPLICA IDENTITY is not FULL — a diagnosis for the original FULL → 42P10 bug that the arbiter registry has since fixed (FULL is now fully supported), so following the old hint led to fixing a non-problem.
• Honour all snapshot modes via a startup planner, and guard a destructive reset against an active replication slot; skip the heartbeat seed for snapshot_only.
• Reject an empty exported-snapshot name before truncating the destination. A destructive snapshot opens the slot, truncates the destination, then copies — but the copy is a no-op when the snapshot handle’s name is empty. An empty name would therefore truncate, copy nothing, checkpoint the empty result as success, and start streaming: silent data loss. BeginSnapshot now fails before the truncate, leaving the destination intact. (It cannot occur against real PostgreSQL, which always exports a snapshot name, but the invariant was previously unguarded.)
• Seed the destination heartbeat row after the snapshot rather than before. In initial + strict mode the snapshot’s COPY of the heartbeat row collided with a pre-seeded row, aborting the first start with a duplicate key.

1.1.4 - 2026-06-02

Fixed

• Wizard-saved configs no longer crash systemd installs with a “read-only file system” error on first start. The setup wizard previously wrote state.boltdb.path: pgpipe-state.db (a relative path). Under systemd’s ProtectSystem=strict, the service’s working directory / is read-only, so the relative path resolved to /pgpipe-state.db and bbolt could not create the file. The unit then restart-looped until StartLimitBurst gave up and the dashboard never came up.

Changed

• pgpipe.service now sets WorkingDirectory=/var/lib/pgpipe (the directory systemd already grants write access to via StateDirectory=pgpipe), and the setup wizard reads $STATE_DIRECTORY from systemd and writes an absolute state.boltdb.path into /etc/pgpipe/pgpipe.yaml. Configs written via the wizard now work whether started by systemd or run by hand.

Upgrade notes

Fresh installs: drop in the v1.1.4 .deb / .rpm. No extra steps.

Already running v1.1.3 or earlier on systemd with a config saved by the setup wizard — the relative path is baked into your pgpipe.yaml. Patch it once and restart:

sudo cp /etc/pgpipe/pgpipe.yaml /etc/pgpipe/pgpipe.yaml.bak
sudo sed -i 's|pgpipe-state\.db|/var/lib/pgpipe/pgpipe-state.db|' \
  /etc/pgpipe/pgpipe.yaml
sudo systemctl reset-failed pgpipe.service
sudo systemctl start pgpipe.service
sudo journalctl -u pgpipe -f      # expect: snapshot complete -> WAL streaming

The workaround is forward-compatible — the absolute path keeps working after you upgrade, so there is nothing to undo.

Docker / manual runs: not affected — no ProtectSystem=strict, the working directory was already writable.

1.1.3 - 2026-05-22

Fixed

• Setup wizard: the Test Connection result now displays and the Next button is correctly enabled after a successful test. The handler had set the result element’s display: none as an inline style before fetching, which left the result hidden and the wizard unable to advance.

1.1.2 - 2026-05-21

Fixed

• Setup wizard error reporting. The test-connection, list-schemas, and list-tables handlers now warn-log failures and info-log successes; connection errors are classified by SQLSTATE / transport and the UI shows a plain-English summary, remediation, and collapsible technical details.
• list-schemas / list-tables surface real errors instead of silently showing “No schemas/tables found”.
• systemd packaging: StartLimitBurst / StartLimitIntervalSec moved to the [Unit] section (fixes the restart-burst loop), and /etc/pgpipe is added to ReadWritePaths and owned pgpipe:pgpipe so the wizard’s Save endpoint can write pgpipe.yaml.

1.1.1 - 2026-05-19

Fixed

• Fix a Go 1.22 net/http.ServeMux pattern panic that crash-looped the setup wizard on startup.

1.1.0 - 2026-05-17

Added

• Opt-in session_replication_role: replica for the destination pool. Skips BEFORE / AFTER ROW triggers and FK re-checks on already-validated replicated rows via a pgxpool AfterConnect hook. Default empty preserves existing behaviour; requires SUPERUSER on the destination.
• fsync_mode knob for the BoltDB state store.
• Writer phase-level Prometheus timing histograms (pgpipe_writer_build_seconds, pgpipe_writer_send_seconds, pgpipe_writer_commit_seconds) for diagnostic visibility into build / send / commit latency.

Changed

• tx_group_size: 50 is now the default and is respected in strict ordering; the writer is op-type aware.
• Parallel-mode throughput scaling: hot-path pipeline counters are now lock-free atomics, removing the single-mutex cap that limited parallel apply to roughly 8 workers.
• Faster writer and snapshot paths: zero-copy string views in the writer hot path, per-row primary-key extraction with a single-table fast path, count(*) replaced with reltuples for snapshot sizing, and dedicated source connections for heartbeat / slot-lag / admin roles.
• Corrected the throughput claim: pgpipe sustains roughly 95% of native PostgreSQL logical-replication apply throughput, with strict per-transaction ordering preserved. Earlier “faster than native” figures are withdrawn.

Deprecated

• Runtime DDL replication is documented as not implemented and is no longer advertised as on-by-default. The internal/ddl package is scaffolding with no production code path: the WAL decoder never emits a DDL event, so source ALTER TABLE statements are not replicated. The destination schema is still created automatically on first run.

Fixed

• The source setup_schema and the heartbeat table it references are now created before the publication references them, instead of failing on a missing relation.

1.0.1 - 2026-04-30

Maintenance build over v1.0.0. There are no source-tracked changes between the v1.0.0 and v1.0.1 builds — see the v1.0.0 entry for the shipped feature set.

1.0.0 - 2026-04-30

First public release. pgpipe streams row changes from one PostgreSQL database to another in real time, with strict per-transaction ordering, a built-in web dashboard, a dead-letter queue, and Prometheus metrics.

Added

• Web dashboard with a login page and a first-run setup wizard.
• Live table management — add and remove replicated tables from the UI, with table-level filtering.
• Dead-letter queue with a REST API: failed events are captured with full context and can be inspected, retried, and replayed; the UI exposes expand / retry row actions.
• Observability: real-time sparkline charts, snapshot progress bars, a WAL-byte replication-lag metric, a stale-data banner, and an SSE log-tail viewer with a config diff / validation preview.
• Pause / resume for the pipeline.
• Prometheus metrics, plus /health and /ready endpoints for liveness and readiness probes.
• HTTPS / TLS for the dashboard via server.tls.cert_file and server.tls.key_file.
• Startup safety checks: PostgreSQL version detection and replica-identity validation, both reported clearly at startup.
• Bidirectional heartbeat verification and replication-slot drift detection.
• .deb and .rpm packages, and cross-compiled binaries for all 8 supported platforms.
• THIRD_PARTY_LICENSES.txt bundled alongside every binary.

Changed

• 5x–10x write throughput via prepared-statement caching and transaction grouping.
• Single-writer strict ordering by default for consistency, with async commit on the destination.

Fixed

• Total-rows counters now survive restarts, and the events-per-second readout no longer shows “idle” while replicating.

Security

• Dashboard authentication is enabled by default, with a random admin password generated and printed on first run.
• Persistent JWT secret so dashboard sessions survive restarts.
• Rate limiting on /api/auth/login, with trusted-proxy X-Forwarded-For handling.
• Security headers on all dashboard responses: CSP, HSTS, X-Frame-Options, X-Content-Type-Options, and Referrer-Policy.