Skip to content

Commit 14e230e

Browse files
Improve Hive Metastore Migration Reliability and Code Quality (#165)
* refactor(hive): improve metastore migration code quality and compatibility - Add proper logging configuration with StreamHandler and formatter - Fix method order in classes by moving __init__ to top - Add compatibility fix for Hive 3.0.0+ by handling CTLG_NAME column - Update AWS regions list with current supported regions - Clean up code style and formatting - Add docstring for _write_dbs_table method explaining Hive compatibility * feat: Reformat the code black8 and isort to be more pythonic. Adding more hive schema changes --------- Co-authored-by: Andor Markus <andor.markus@allcloud.io>
1 parent e4faa2d commit 14e230e

File tree

2 files changed

+887
-709
lines changed

2 files changed

+887
-709
lines changed

utilities/Hive_metastore_migration/src/export_from_datacatalog.py

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -33,11 +33,8 @@ def change_schemas(sql_context, databases, tables, partitions):
3333

3434

3535
def datacatalog_migrate_to_hive_metastore(sc, sql_context, databases, tables, partitions, connection):
36-
3736
hive_metastore = HiveMetastore(connection, sql_context)
38-
3937
transform_databases_tables_partitions(sc, sql_context, hive_metastore, databases, tables, partitions)
40-
4138
hive_metastore.export_to_metastore()
4239

4340

0 commit comments

Comments
 (0)