Flowsight configuration for PostgreSQL
PostgreSQL is one of Snowflake Openflow's own built-in connectors — it ships with Openflow itself. Flowsight drives it for you: you set it up through the four-step wizard and watch it from the monitoring dashboard, without ever touching the NiFi canvas.
Flowsight replicates tables from a PostgreSQL database into Snowflake. It performs an initial bulk snapshot load of each selected table, then keeps the destination current through continuous incremental sync — change data capture of inserts, updates, and deletes, read from PostgreSQL's logical-replication stream.
What you can do with it
- Full replication — mirror a database's tables into a Snowflake schema and keep them continuously up to date.
- Selective replication — replicate only the tables you list in a publication.
- Analytics over operational data — land application data in Snowflake to query, join, and model alongside other warehouse data, without loading the source database.
How it works
- Publication — you create a PostgreSQL publication naming the tables to replicate. A publication is a logical-replication object: a named set of tables your database agrees to stream row-level changes for. The connector subscribes to it by name.
- Snapshot load — each table's current contents are bulk-loaded into the destination.
- Incremental sync — after the snapshot, row-level changes flow from the publication and are merged into the destination tables continuously.
Set it up
Pick Set up on the PostgreSQL card. The wizard walks four steps:
- Prerequisites — confirm your runtime and source access.
- Select runtime — bind the Openflow runtime that will run the connector.
- Connect endpoints — enter the source and destination details below.
- Select tables — choose which tables to replicate.
When you finish, the sync starts and the Monitoring tab lights up.
Source
| Field | What it is |
|---|---|
| Connection URL | The JDBC URL of the source database. |
| Username | A database user the connector authenticates as. |
| Password | That user's password. |
| Publication name | The publication you created listing the tables to replicate. |
Destination
| Field | What it is |
|---|---|
| Database | An existing Snowflake database to load into. |
| Schema | An existing schema to load every table into, or blank to mirror source names. |
| Role | The role your runtime executes as (the "Execute-as role" from runtime creation). |
| Warehouse | The warehouse that runs the journal-to-destination merges. |
Before you start: you need an Openflow runtime, network access from the runtime to your database on port
5432(Flowsight opens this egress for you), a PostgreSQL publication with logical replication enabled, and a Snowflake database, schema, and warehouse for the load.
How your data lands
Replicated tables land in your chosen <database>.<schema> and carry two
columns Flowsight manages:
| Column | Snowflake type | Purpose |
|---|---|---|
_SNOWFLAKE_DELETED | BOOLEAN | Soft-delete tombstone flag. |
_SNOWFLAKE_UPDATED_AT | TIMESTAMP_NTZ | When the row was last applied. |
Query the replicated data
Deletes are soft deletes — the row stays and _SNOWFLAKE_DELETED is set to
TRUE. Filter them out to get the live set:
SELECT *
FROM <database>.<schema>.<table>
WHERE _SNOWFLAKE_DELETED = FALSE;
Adding and removing tables
The set of replicated tables is governed by your publication. To add a table, add it to the publication on the source database; to stop replicating one, remove it from the publication. Removing a table stops further updates to its destination table — it does not drop the table or its data from Snowflake.
Notes
- Logical replication required — the source database must have logical
replication enabled, and creating a publication requires a role with the
REPLICATIONattribute. - Egress on port 5432 — the connect step pre-warms the runtime's egress rule to your database host so replication can connect without a propagation delay.