Skip to content

Commit d6da015

Browse files
committed
CFR: Truncate target table before importing
Also, use `append` strategy again, because `replace` doesn't do the right DDL.
1 parent 13d2739 commit d6da015

File tree

2 files changed

+8
-3
lines changed

2 files changed

+8
-3
lines changed

CHANGES.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,8 +4,10 @@
44
- I/O: Updated to `ingestr>=0.13.61`
55
- CFR: Improved log output
66
- CFR: Fixed double quoting of table name. Thanks, @karynzv.
7-
- CFR: When importing, use `replace` policy instead of `append`
7+
- CFR: When importing, started using `replace` strategy instead of `append`
88
- CFR: Improved importing data re. type mapping without NumPy
9+
- CFR: Truncated target table before importing, using `append`
10+
strategy again, because `replace` doesn't do the right DDL.
911

1012
## 2025/07/01 v0.0.37
1113
- Settings: Fixed comparison of `0s` vs `0ms`. Thanks, @hlcianfagna.

cratedb_toolkit/cfr/systable.py

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -234,7 +234,7 @@ def load(self):
234234
def _load(self, path_schema: Path, path_data: Path):
235235
table_count = 0
236236
for tablename in tqdm(self.table_names()):
237-
tablename_restored = ExportSettings.TABLE_FILENAME_PREFIX + tablename
237+
tablename_restored = f"{ExportSettings.TABLE_FILENAME_PREFIX}{tablename}"
238238

239239
path_table_schema = path_schema / f"{ExportSettings.TABLE_FILENAME_PREFIX}{tablename}.sql"
240240
path_table_data = path_data / f"{ExportSettings.TABLE_FILENAME_PREFIX}{tablename}.{self.data_format}"
@@ -249,14 +249,17 @@ def _load(self, path_schema: Path, path_data: Path):
249249
schema_sql = path_table_schema.read_text()
250250
self.adapter.run_sql(schema_sql)
251251

252+
# Truncate table.
253+
self.adapter.run_sql(f"DELETE FROM {self.adapter.quote_relation_name(tablename_restored)};") # noqa: S608
254+
252255
# Load data.
253256
try:
254257
df: "pd.DataFrame" = pd.DataFrame.from_records(self.load_table(path_table_data))
255258
df.to_sql(
256259
name=tablename_restored,
257260
con=self.adapter.engine,
258261
index=False,
259-
if_exists="replace",
262+
if_exists="append",
260263
method=insert_bulk,
261264
)
262265
except Exception as ex:

0 commit comments

Comments
 (0)