build profiling image in CI, add arm64 support to AWS notebooks #68
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
More miscellaneous refactoring.
windows/
directory from LightGBM (no longer necessary to build the package, as of [python-package] remove MSVS solution files from sdist microsoft/LightGBM#6698)dask_cloudprovider.FargateCluster
example, to fix this:LightGBM/
as read-write at build timeNotes for Reviewers
Why set CPU architecture?
This repo only builds single-architecture container images.
The notebooks are intended to be run locally, from the same machine where you built all the images.
dask_cloudprovider.FargateCluster
defaults to x86_64.When I built the images on my M2 Mac (which is arm64) then tried to run the
demo-aws
notebook as-is, I saw this error on ECS:This fixes that by detecting CPU architecture on the machine where the notebook is running, and passing that through. Thanks to @dmitry-livchak for adding that support in dask/dask-cloudprovider#425 😁
Why make
LightGBM/
read-write at build time?Without it, the build fails like this:
Because
build-python.sh
from LightGBM creates a temporary directory and moves files into it (code link).