Skip to content

Conversation

@leaves12138
Copy link
Contributor

Purpose

Linked issue: close #xxx

Tests

API and Format

Documentation

@leaves12138 leaves12138 changed the title [WIP] [python] Remove all redundant change in pr 6969 [WIP] [python] Remove all redundant changes in pr 6969 Jan 8, 2026
@leaves12138 leaves12138 changed the title [WIP] [python] Remove all redundant changes in pr 6969 [python] Remove all redundant changes in pr 6969 Jan 8, 2026
# Generate partition statistics for the commit
statistics = self._generate_partition_statistics(commit_entries)
except Exception as e:
self._cleanup_preparation_failure(new_manifest_file, delta_manifest_list,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Keep this as it is.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK

result = self._try_commit_once(
retry_result=retry_result,
commit_kind=commit_kind,
commit_entries=commit_entries,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pass a function, overwrite commit should generate commit_entries by snapshot.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK

@leaves12138 leaves12138 changed the title [python] Remove all redundant changes in pr 6969 [WIP] [python] Remove all redundant changes in pr 6969 Jan 8, 2026
+ f"with identifier {commit_identifier} and kind {commit_kind}."
)
return SuccessResult()
def _cleanup_preparation_failure(self, manifest_file: Optional[str],
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why pass manifest_file? delta_manifest_list already contains this file?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed

@leaves12138 leaves12138 changed the title [WIP] [python] Remove all redundant changes in pr 6969 [python] Remove all redundant changes in pr 6969 Jan 8, 2026
latest_snapshot: Optional[Snapshot]) -> CommitResult:
start_time_ms = int(time.time() * 1000)

if retry_result is not None and latest_snapshot is not None:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Keep this as it is, and add test to verify this.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure

commit_entries=None, # Will be generated in _try_commit based on latest snapshot
commit_identifier=commit_identifier
commit_identifier=commit_identifier,
commit_entries_plan=lambda snapshot: self._generate_overwrite_entries(snapshot)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove self._overwrite_partition_filter and self._overwrite_commit_messages.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK

latest_snapshot: Optional[Snapshot]) -> CommitResult:
start_time_ms = int(time.time() * 1000)

if retry_result is not None and latest_snapshot is not None:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

extract a method for this.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK

if retry_result.latest_snapshot is not None:
start_check_snapshot_id = retry_result.latest_snapshot.id + 1

for snapshot_id in range(start_check_snapshot_id, latest_snapshot.id + 2):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+2?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1!

@JingsongLi
Copy link
Contributor

+1

@JingsongLi JingsongLi merged commit 470b381 into apache:master Jan 8, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants