-
Notifications
You must be signed in to change notification settings - Fork 1.2k
[python] Remove all redundant changes in pr 6969 #6977
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
| # Generate partition statistics for the commit | ||
| statistics = self._generate_partition_statistics(commit_entries) | ||
| except Exception as e: | ||
| self._cleanup_preparation_failure(new_manifest_file, delta_manifest_list, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Keep this as it is.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OK
| result = self._try_commit_once( | ||
| retry_result=retry_result, | ||
| commit_kind=commit_kind, | ||
| commit_entries=commit_entries, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
pass a function, overwrite commit should generate commit_entries by snapshot.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OK
| + f"with identifier {commit_identifier} and kind {commit_kind}." | ||
| ) | ||
| return SuccessResult() | ||
| def _cleanup_preparation_failure(self, manifest_file: Optional[str], |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why pass manifest_file? delta_manifest_list already contains this file?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Indeed
| latest_snapshot: Optional[Snapshot]) -> CommitResult: | ||
| start_time_ms = int(time.time() * 1000) | ||
|
|
||
| if retry_result is not None and latest_snapshot is not None: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Keep this as it is, and add test to verify this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sure
| commit_entries=None, # Will be generated in _try_commit based on latest snapshot | ||
| commit_identifier=commit_identifier | ||
| commit_identifier=commit_identifier, | ||
| commit_entries_plan=lambda snapshot: self._generate_overwrite_entries(snapshot) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
remove self._overwrite_partition_filter and self._overwrite_commit_messages.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OK
| latest_snapshot: Optional[Snapshot]) -> CommitResult: | ||
| start_time_ms = int(time.time() * 1000) | ||
|
|
||
| if retry_result is not None and latest_snapshot is not None: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
extract a method for this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OK
| if retry_result.latest_snapshot is not None: | ||
| start_check_snapshot_id = retry_result.latest_snapshot.id + 1 | ||
|
|
||
| for snapshot_id in range(start_check_snapshot_id, latest_snapshot.id + 2): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
+2?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
+1!
|
+1 |
Purpose
Linked issue: close #xxx
Tests
API and Format
Documentation