Skip to content

Commit e44d7e7

Browse files
refactor: Deprecate sqlalchemy_url and dialect+driver settings (#482)
1 parent 41a9d99 commit e44d7e7

File tree

5 files changed

+60
-65
lines changed

5 files changed

+60
-65
lines changed

README.md

Lines changed: 26 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -24,34 +24,32 @@ This target is tested with all actively supported [Python](https://devguide.pyth
2424

2525
## Settings
2626

27-
| Setting | Required | Default | Description |
28-
| :------------------------------ | :------- | :---------------------------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
29-
| host | False | None | Hostname for postgres instance. Note if sqlalchemy_url is set this will be ignored. |
30-
| port | False | 5432 | The port on which postgres is awaiting connection. Note if sqlalchemy_url is set this will be ignored. |
31-
| user | False | None | User name used to authenticate. Note if sqlalchemy_url is set this will be ignored. |
32-
| password | False | None | Password used to authenticate. Note if sqlalchemy_url is set this will be ignored. |
33-
| database | False | None | Database name. Note if sqlalchemy_url is set this will be ignored. |
34-
| sqlalchemy_url | False | None | SQLAlchemy connection string. This will override using host, user, password, port, dialect, and all ssl settings. Note that you must escape password special characters properly. See https://docs.sqlalchemy.org/en/20/core/engines.html#escaping-special-characters-such-as-signs-in-passwords |
35-
| dialect+driver | False | postgresql+psycopg | Dialect+driver see https://docs.sqlalchemy.org/en/20/core/engines.html. Generally just leave this alone. Note if sqlalchemy_url is set this will be ignored. |
36-
| default_target_schema | False | melty | Postgres schema to send data to, example: tap-clickup |
37-
| activate_version | False | 1 | If set to false, the tap will ignore activate version messages. If set to true, add_record_metadata must be set to true as well. |
38-
| hard_delete | False | 0 | When activate version is sent from a tap this specefies if we should delete the records that don't match, or mark them with a date in the `_sdc_deleted_at` column. This config option is ignored if `activate_version` is set to false. |
39-
| add_record_metadata | False | 1 | Note that this must be enabled for activate_version to work!This adds _sdc_extracted_at, _sdc_batched_at, and more to every table. See https://sdk.meltano.com/en/latest/implementation/record_metadata.html for more information. |
40-
| interpret_content_encoding | False | 0 | If set to true, the target will interpret the content encoding of the schema to determine how to store the data. Using this option may result in a more efficient storage of the data but may also result in an error if the data is not encoded as expected. |
41-
| ssl_enable | False | 0 | Whether or not to use ssl to verify the server's identity. Use ssl_certificate_authority and ssl_mode for further customization. To use a client certificate to authenticate yourself to the server, use ssl_client_certificate_enable instead. Note if sqlalchemy_url is set this will be ignored. |
42-
| ssl_client_certificate_enable | False | 0 | Whether or not to provide client-side certificates as a method of authentication to the server. Use ssl_client_certificate and ssl_client_private_key for further customization. To use SSL to verify the server's identity, use ssl_enable instead. Note if sqlalchemy_url is set this will be ignored. |
43-
| ssl_mode | False | verify-full | SSL Protection method, see [postgres documentation](https://www.postgresql.org/docs/current/libpq-ssl.html#LIBPQ-SSL-PROTECTION) for more information. Must be one of disable, allow, prefer, require, verify-ca, or verify-full. Note if sqlalchemy_url is set this will be ignored. |
44-
| ssl_certificate_authority | False | ~/.postgresql/root.crl | The certificate authority that should be used to verify the server's identity. Can be provided either as the certificate itself (in .env) or as a filepath to the certificate. Note if sqlalchemy_url is set this will be ignored. |
45-
| ssl_client_certificate | False | ~/.postgresql/postgresql.crt | The certificate that should be used to verify your identity to the server. Can be provided either as the certificate itself (in .env) or as a filepath to the certificate. Note if sqlalchemy_url is set this will be ignored. |
46-
| ssl_client_private_key | False | ~/.postgresql/postgresql.key | The private key for the certificate you provided. Can be provided either as the certificate itself (in .env) or as a filepath to the certificate. Note if sqlalchemy_url is set this will be ignored. |
47-
| ssl_storage_directory | False | .secrets | The folder in which to store SSL certificates provided as raw values. When a certificate/key is provided as a raw value instead of as a filepath, it must be written to a file before it can be used. This configuration option determines where that file is created. |
48-
| ssh_tunnel | False | None | SSH Tunnel Configuration, this is a json object |
49-
| ssh_tunnel.enable | False | 0 | Enable an ssh tunnel (also known as bastion host), see the other ssh_tunnel.* properties for more details |
50-
| ssh_tunnel.host | False | None | Host of the bastion host, this is the host we'll connect to via ssh |
51-
| ssh_tunnel.username | False | None | Username to connect to bastion host |
52-
| ssh_tunnel.port | False | 22 | Port to connect to bastion host |
53-
| ssh_tunnel.private_key | False | None | Private Key for authentication to the bastion host |
54-
| ssh_tunnel.private_key_password | False | None | Private Key Password, leave None if no password is set |
27+
| Setting | Required | Default | Description |
28+
| :------------------------------ | :------- | :--------------------------- | :--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
29+
| host | False | None | Hostname for postgres instance. |
30+
| port | False | 5432 | The port on which postgres is awaiting connections. |
31+
| user | False | None | User name used to authenticate. |
32+
| password | False | None | Password used to authenticate. |
33+
| database | False | None | Database name. |
34+
| default_target_schema | False | melty | Postgres schema to send data to, example: tap-clickup |
35+
| activate_version | False | 1 | If set to false, the tap will ignore activate version messages. If set to true, add_record_metadata must be set to true as well. |
36+
| hard_delete | False | 0 | When activate version is sent from a tap this specefies if we should delete the records that don't match, or mark them with a date in the `_sdc_deleted_at` column. This config option is ignored if `activate_version` is set to false. |
37+
| add_record_metadata | False | 1 | Note that this must be enabled for activate_version to work!This adds _sdc_extracted_at, _sdc_batched_at, and more to every table. See https://sdk.meltano.com/en/latest/implementation/record_metadata.html for more information. |
38+
| interpret_content_encoding | False | 0 | If set to true, the target will interpret the content encoding of the schema to determine how to store the data. Using this option may result in a more efficient storage of the data but may also result in an error if the data is not encoded as expected. |
39+
| ssl_enable | False | 0 | Whether or not to use ssl to verify the server's identity. Use ssl_certificate_authority and ssl_mode for further customization. To use a client certificate to authenticate yourself to the server, use ssl_client_certificate_enable instead. |
40+
| ssl_client_certificate_enable | False | 0 | Whether or not to provide client-side certificates as a method of authentication to the server. Use ssl_client_certificate and ssl_client_private_key for further customization. To use SSL to verify the server's identity, use ssl_enable instead. |
41+
| ssl_mode | False | verify-full | SSL Protection method, see [postgres documentation](https://www.postgresql.org/docs/current/libpq-ssl.html#LIBPQ-SSL-PROTECTION) for more information. Must be one of disable, allow, prefer, require, verify-ca, or verify-full. |
42+
| ssl_certificate_authority | False | ~/.postgresql/root.crl | The certificate authority that should be used to verify the server's identity. Can be provided either as the certificate itself (in .env) or as a filepath to the certificate. |
43+
| ssl_client_certificate | False | ~/.postgresql/postgresql.crt | The certificate that should be used to verify your identity to the server. Can be provided either as the certificate itself (in .env) or as a filepath to the certificate. |
44+
| ssl_client_private_key | False | ~/.postgresql/postgresql.key | The private key for the certificate you provided. Can be provided either as the certificate itself (in .env) or as a filepath to the certificate. |
45+
| ssl_storage_directory | False | .secrets | The folder in which to store SSL certificates provided as raw values. When a certificate/key is provided as a raw value instead of as a filepath, it must be written to a file before it can be used. This configuration option determines where that file is created. |
46+
| ssh_tunnel | False | None | SSH Tunnel Configuration, this is a json object |
47+
| ssh_tunnel.enable | False | 0 | Enable an ssh tunnel (also known as bastion host), see the other ssh_tunnel.* properties for more details |
48+
| ssh_tunnel.host | False | None | Host of the bastion host, this is the host we'll connect to via ssh |
49+
| ssh_tunnel.username | False | None | Username to connect to bastion host |
50+
| ssh_tunnel.port | False | 22 | Port to connect to bastion host |
51+
| ssh_tunnel.private_key | False | None | Private Key for authentication to the bastion host |
52+
| ssh_tunnel.private_key_password | False | None | Private Key Password, leave None if no password is set |
5553

5654
A full list of supported settings and capabilities is available by running: `target-postgres --about`
5755

meltano.yml

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -35,9 +35,6 @@ plugins:
3535
namespace: target_postgres
3636
pip_url: -e .
3737
settings:
38-
- name: sqlalchemy_url
39-
kind: password
40-
sensitive: true
4138
- name: ssl_enable
4239
kind: boolean
4340
sensitive: true

target_postgres/driver.py

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
"""This module contains the Postgres driver name for SQLAlchemy."""
2+
3+
PSYCOPG3 = "postgresql+psycopg"

0 commit comments

Comments
 (0)