Workflows to build python CLI release and pypi artifacts #3036
Conversation
HonahX
changed the title
| logging.basicConfig(level=logging.INFO, format="%(message)s") | ||
|
|
||
| if os.environ.get("POLARIS_CLI_SKIP_CLIENT_GENERATION", "false").lower() == "true": | ||
| logger.info("POLARIS_CLI_SKIP_CLIENT_GENERATION is set to true, skipping client generation.") |
There was a problem hiding this comment.
The cibuildwheel environment does not have java, so we add this to skip the generation there
| # Use test values for pull_request trigger, actual inputs for workflow_dispatch | ||
| if [[ "${{ github.event_name }}" == "pull_request" ]]; then | ||
| echo "Workflow triggered by pull_request - using test values" | ||
| VERSION="1.2.0" |
There was a problem hiding this comment.
NIT: Not sure if we should use 1.2.0 as the default version during test. Maybe we should do 0.0.0 instead to avoid confusion?
There was a problem hiding this comment.
Hello 👋 Just want to highlight that in this dev@ thread was discussed legal aspects and versioning. I think it's better to sync with JB and other engineers who has expertise.
There was a problem hiding this comment.
Hi @MonkeyCanCode @binarycat0 This aims to be an example version when the workflow is triggered by pull request to verify if CLI changes break the wheel build
When it comes to real release, the release manager will need to manually enter the version and rc number to build the release artifacts. And the workflow will verify if the version is consistent with the one in the package metadata For example (pyiceberg)
And of course, in the future it could be integrated into release automation pipeline
|
@HonahX very nice work. We are almost there to have package in PyPI. |
snazy
left a comment
snazy
left a comment
There was a problem hiding this comment.
Is there a particular reason to not extent the release-automation workflows?
We have to follow the ASF release policy. It seems there are some steps missing in this PR:
- checksums
- gpg signatures
- RC staging
- staging the source tarball and artifacts to vote on
- eventually publishing exactly the artifacts that have been voted on
|
Hi @snazy Thanks for highlighting these important elements of the final release candidate for python CLI. Given the number of missing pieces in the CLI release process, my goal is to separate the pipeline into 2 milestone:
This PR aims to focus on the first milestone in order to have additional CI to ensure that all changes are compatible with the release process and unblock nightly build. The signature/checksum/release candidate are goals in the second milestone. This way we could first determine what to release and start testing the artifacts and then finishing the rest part. WDYT? |
4 participants
This PR include github workflow, "Python Client Build Release Candidate" to build release artifacts for polaris python CLI. Currently anyone with write permission could manually trigger the workflow and specifying the version and rc version, and the workflow will give you 2 artifact
python-client-pypi-release-candidate-1.2.0rc1.zippython-client-svn-release-candidate-1.2.0rc1.zipEach artifact will contain sdist and wheels built for macos and linux (TODO: support windows)
Note the only difference between pypi and svn artifact is that the pypi version will have the
rc1attached to the package version, so that it could be uploaded to pypi as a "pre-release" version. The svn artifact should be the one to vote and publish finally.This workflow pipeline is modified from https://github.com/apache/iceberg-python/blob/main/.github/workflows/python-release.yml, great thanks to @kevinjqliu who was the original author of that.
cc: @binarycat0 @MonkeyCanCode @flyrain @snazy
Checklist
CHANGELOG.md(if needed)site/content/in-dev/unreleased(if needed)