Skip to content

Commit 19845b7

Browse files
docs: Update README to mention replication_slot_name (#630)
1 parent 686dfdc commit 19845b7

File tree

1 file changed

+21
-0
lines changed

1 file changed

+21
-0
lines changed

README.md

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,7 @@ This tap is tested with all actively supported [Python](https://devguide.python.
2727
| password | False | None | Password used to authenticate. Note if sqlalchemy_url is set this will be ignored. |
2828
| database | False | None | Database name. Note if sqlalchemy_url is set this will be ignored. |
2929
| max_record_count | False | None | Optional. The maximum number of records to return in a single stream. |
30+
| replication_slot_name | False | tappostgres | Optional. Name of the replication slot created with `pg_create_logical_replication_slot` |
3031
| sqlalchemy_url | False | None | Example postgresql://[username]:[password]@localhost:5432/[db_name] |
3132
| filter_schemas | False | None | If an array of schema names is provided, the tap will only process the specified Postgres schemas and ignore others. If left blank, the tap automatically determines ALL available Postgres schemas. |
3233
| dates_as_string | False | 0 | Defaults to false, if true, date, and timestamp fields will be Strings. If you see ValueError: Year is out of range, try setting this to True. |
@@ -267,44 +268,64 @@ Note also that using log-based replication will cause the replication key for al
267268
1. Ensure you are using PostgresSQL 9.4 or higher.
268269
1. Need to access the master postgres instance
269270
1. Install the wal2json plugin for your database. Example instructions are given below for a Postgres 15.0 database running on Ubuntu 22.04. For more information, or for alternative versions/operating systems, refer to the [wal2json documentation](https://github.com/eulerto/wal2json)
271+
270272
- Update and upgrade apt if necessary.
273+
271274
```bash
272275
sudo apt update
273276
sudo apt upgrade -y
274277
```
278+
275279
- Prepare by making prerequisite installations.
280+
276281
```bash
277282
sudo apt install curl ca-certificates
278283
sudo install -d /usr/share/postgresql-common/pgdg
279284
```
285+
280286
- Import the repository keys for the Postgres Apt repository
287+
281288
```bash
282289
sudo curl -o /usr/share/postgresql-common/pgdg/apt.postgresql.org.asc --fail https://www.postgresql.org/media/keys/ACCC4CF8.asc
283290
```
291+
284292
- Create the pgdg.list file.
293+
285294
```bash
286295
sudo sh -c 'echo "deb [signed-by=/usr/share/postgresql-common/pgdg/apt.postgresql.org.asc] https://apt.postgresql.org/pub/repos/apt bookworm-pgdg main" > /etc/apt/sources.list.d/pgdg.list'
287296
```
297+
288298
- Use the Postgres Apt repository to install wal2json
299+
289300
```bash
290301
sudo apt update
291302
sudo apt-get install postgresql-server-dev-15
292303
export PATH=/usr/lib/postgresql/15/bin:$PATH
293304
sudo apt-get install postgresql-15-wal2json
294305
```
306+
295307
1. Configure your database with wal2json enabled.
308+
296309
- Edit your `postgresql.conf` configuration file so the following parameters are appropriately set.
310+
297311
```
298312
wal_level = logical
299313
max_replication_slots = 10
300314
max_wal_senders = 10
301315
```
316+
302317
- Restart PostgresSQL
318+
303319
- Create a replication slot for tap-postgres.
320+
304321
```sql
305322
SELECT * FROM pg_create_logical_replication_slot('tappostgres', 'wal2json');
306323
```
324+
325+
You can also create the replication slot with any name you want, and set the `replication_slot_name` setting to that value.
326+
307327
1. Ensure your configuration for tap-postgres specifies host, port, user, password, and database manually, without relying on an sqlalchemy url.
328+
308329
1. Use the following metadata modification in your `meltano.yml` for the streams you wish to have as log-based. Note that during log-based replication, we do not support any replication key other than `_sdc_lsn`.
309330
```yml
310331
metadata:

0 commit comments

Comments
 (0)