Skip to content

Conversation

@nickdelnano
Copy link

This PR adds a new format for defining table properties as specific.table.properties.<database_name>.<table_name>.<property>. This allows defining properties that are specific to a table. The table specific property takes precedence.

Paimon sink supports defining table properties as table.properties.*. They apply to ALL tables. In some cases it is warranted to set table properties based on the table use case. Some examples are compaction settings, Iceberg compatibility settings, etc. This PR allows this.

@nickdelnano nickdelnano changed the title Paimon connector - support table specific table.properties config [pipeline-connector/paimon] Support table specific table.properties configuration Aug 6, 2025
@nickdelnano
Copy link
Author

@lvyanquan can you review this change?

@lvyanquan
Copy link
Contributor

We can modify the table options through transform, but I agree to provide more configurations in the sink as alternative solutions, refer to https://nightlies.apache.org/flink/flink-cdc-docs-release-3.4/docs/core-concept/transform/#specify-table-creation-configuration.

Compared to transform, the current configuration requires too many parameters to be set for situations where many tables need to be parameterized(users usually synchronize a large number of tables in one job). Can we provide a solution for parameter configuration based on regular expressions (Selectors)?

@github-actions
Copy link

github-actions bot commented Dec 9, 2025

This pull request has been automatically marked as stale because it has not had recent activity for 120 days. It will be closed in 60 days if no further activity occurs.

@github-actions github-actions bot added the Stale label Dec 9, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants