[Connector/JDBC] Support passing arbitrary database options to JDBC Catalog #183
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
1. Motivation
Currently, when creating a
JDBC Catalogin Flink SQL, there is no way to pass arbitrary connectionpropertiesto the JDBC driver. This is a significant limitation compared toJDBC Dynamic Tables, which support a generic properties option.This missing feature can lead to connection failures for certain databases that require specific driver properties. A common example is connecting to a MySQL or MariaDB server with a non-UTC timezone (e.g., 'KST'), which results in the following error:
To resolve this, the user needs to be able to pass properties like
serverTimezone=UTC. Similarly, other use cases, such as passingstringtype=unspecifiedfor PostgreSQL, are not possible with the current JDBC Catalog implementation.Previous attempts to address this (e.g., PR #74, PR #83) were initiated but not completed. This PR provides a complete and robust solution to this long-standing issue.
2. Solution
This PR introduces a new configuration option,
database-options, to theJdbcCatalogFactory. This option allows users to specify a string of key-value pairs that will be appended to the JDBC connection URL.This approach provides a flexible and universal way to configure any required JDBC driver property without adding database-specific options.
3. Implementation Details
The changes were implemented as follows:
JdbcCatalogFactory.java:
A new ConfigOption named
database-optionswas added.This option is registered in
optionalOptions()to make it non-mandatory.AbstractJdbcCatalog.java:
The
getDatabaseUrl()method was modified to check for the presence ofdatabase-options.If the options exist, they are appended to the base JDBC URL using the correct separator (
?or;based on the database dialect).Factories and Loaders:
The database-options value is now passed through the factory chain: JdbcCatalogFactory -> JdbcFactoryLoader -> database-specific factories (MySqlFactory, PostgresFactory, etc.).
The method signatures in JdbcFactory.java and JdbcFactoryLoader.java were overloaded to accept the new dbOptions parameter.
Database-Specific Implementations:
All existing database-specific catalog implementations (MySQL, PostgreSQL, Derby, OceanBase, etc.) and their corresponding factories were updated to accept and pass the dbOptions parameter to the AbstractJdbcCatalog constructor.
Usage Example
With this change, a user can now create a JDBC catalog for MySQL and specify the required timezone property as follows:
Verification
The changes have been tested locally by creating a JDBC catalog for a MariaDB database that requires the serverTimezone property.
mysql
oceanbase
postgresql
Related Issues/PRs
JIRA: FLINK-38616
Stale PRs: [FLINK-33069]Mysql and Postgres catalog support url extra parameters #74, [FLINK-33800][JDBC/Connector] Allow passing parameters to database via jdbc url #83