Skip to content

Commit c0b52ff

Browse files
dbschmigelskidbschmigelski
committed
ci: add workflow for lambda layer publish and yank
1 parent 293f00e commit c0b52ff

File tree

3 files changed

+223
-87
lines changed

3 files changed

+223
-87
lines changed

‎.github/workflows/LAMBDA_LAYERS_SOP.md‎

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
# Lambda Layers Standard Operating Procedures (SOP)
2+
3+
## Overview
4+
5+
This document defines the standard operating procedures for managing Strands Agents Lambda layers across all AWS regions, Python versions, and architectures.
6+
7+
**Total: 136 individual Lambda layers** (17 regions × 2 architectures × 4 Python versions). All variants must maintain the same layer version number for each PyPI package version, with only one row per PyPI version appearing in documentation.
8+
9+
## Deployment Process
10+
11+
### 1. Initial Deployment
12+
1. Run workflow with ALL options selected (default)
13+
2. Specify PyPI package version
14+
3. Type "Create Lambda Layer {package_version}" to confirm
15+
4. All 136 individual layers deploy in parallel (4 Python × 2 arch × 17 regions)
16+
5. Each layer gets its own unique name: `strands-agents-py{PYTHON_VERSION}-{ARCH}`
17+
18+
### 2. Version Buffering for New Variants
19+
When adding new variants (new Python version, architecture, or region):
20+
21+
1. **Determine target layer version**: Check existing variants to find the highest layer version
22+
2. **Buffer deployment**: Deploy new variants multiple times until layer version matches existing variants
23+
3. **Example**: If existing variants are at layer version 5, deploy new variant 5 times to reach version 5
24+
25+
### 3. Handling Transient Failures
26+
When some regions fail during deployment:
27+
28+
1. **Identify failed regions**: Check which combinations didn't complete successfully
29+
2. **Targeted redeployment**: Use specific region/arch/Python inputs to redeploy failed combinations
30+
3. **Version alignment**: Continue deploying until all variants reach the same layer version
31+
4. **Verification**: Confirm all combinations have identical layer versions before updating docs
32+
33+
## Yank Process
34+
35+
### Yank Procedure
36+
1. Use the `yank_lambda_layer` GitHub action workflow
37+
2. Specify the layer version to yank
38+
3. Type "Yank Lambda Layer {layer_version}" to confirm
39+
4. **Full yank**: Run with ALL options selected (default) to yank all 136 variants OR **Partial yank**: Specify Python versions, architectures, and regions for targeted yanking
40+
6. Update documentation
41+
7. **Communication**: Notify users through appropriate channels
42+
43+
**Note**: Yanking deletes layer versions completely. Existing Lambda functions using the layer continue to work, but new functions cannot use the yanked version.

‎.github/workflows/publish-lambda-layer.yml‎

Lines changed: 99 additions & 87 deletions
Original file line numberDiff line numberDiff line change
@@ -9,64 +9,90 @@ on:
99
type: string
1010
python_version:
1111
description: 'Python version'
12-
required: false
13-
default: '3.12'
12+
required: true
13+
default: 'ALL'
1414
type: choice
15-
options: ['3.10', '3.11', '3.12', '3.13']
15+
options: ['ALL', '3.10', '3.11', '3.12', '3.13']
1616
architecture:
1717
description: 'Architecture'
18-
required: false
19-
default: 'x86_64'
18+
required: true
19+
default: 'ALL'
2020
type: choice
21-
options: ['x86_64', 'aarch64']
21+
options: ['ALL', 'x86_64', 'aarch64']
2222
region:
2323
description: 'AWS region'
24-
required: false
25-
default: 'us-east-1'
24+
required: true
25+
default: 'ALL'
2626
type: choice
2727
# Only non opt-in regions included for now
28-
options: ['us-east-1', 'us-east-2', 'us-west-1', 'us-west-2', 'ap-south-1', 'ap-northeast-1', 'ap-northeast-2', 'ap-northeast-3', 'ap-southeast-1', 'ap-southeast-2', 'ca-central-1', 'eu-central-1', 'eu-west-1', 'eu-west-2', 'eu-west-3', 'eu-north-1', 'sa-east-1']
28+
options: ['ALL', 'us-east-1', 'us-east-2', 'us-west-1', 'us-west-2', 'ap-south-1', 'ap-northeast-1', 'ap-northeast-2', 'ap-northeast-3', 'ap-southeast-1', 'ap-southeast-2', 'ca-central-1', 'eu-central-1', 'eu-west-1', 'eu-west-2', 'eu-west-3', 'eu-north-1', 'sa-east-1']
2929
confirm:
30-
description: 'Type "Create Lambda Layer" to confirm publishing the layer'
30+
description: 'Type "Create Lambda Layer {PyPI version}" to confirm publishing the layer'
3131
required: true
3232
type: string
3333

3434
env:
35-
IS_FULL_DEPLOY: ${{ !inputs.python_version && !inputs.architecture && !inputs.region }}
35+
BUCKET_NAME: strands-agents-lambda-layer
3636

3737
jobs:
38-
publish-layer:
38+
validate:
39+
runs-on: ubuntu-latest
40+
steps:
41+
- name: Validate confirmation
42+
run: |
43+
CONFIRM="${{ inputs.confirm }}"
44+
EXPECTED="Create Lambda Layer ${{ inputs.package_version }}"
45+
if [ "$CONFIRM" != "$EXPECTED" ]; then
46+
echo "Confirmation failed. You must type exactly '$EXPECTED' to proceed."
47+
exit 1
48+
fi
49+
echo "Confirmation validated"
50+
51+
create-buckets:
52+
needs: validate
3953
runs-on: ubuntu-latest
4054
strategy:
4155
matrix:
42-
python-version: ${{ inputs.python_version && fromJson(format('["{}"', inputs.python_version)) || fromJson('["3.10", "3.11", "3.12", "3.13"]') }}
43-
architecture: ${{ inputs.architecture && fromJson(format('["{}"', inputs.architecture)) || fromJson('["x86_64", "aarch64"]') }}
44-
region: ${{ inputs.region && fromJson(format('["{}"', inputs.region)) || fromJson('["us-east-1", "us-east-2", "us-west-1", "us-west-2", "ap-south-1", "ap-northeast-1", "ap-northeast-2", "ap-northeast-3", "ap-southeast-1", "ap-southeast-2", "ca-central-1", "eu-central-1", "eu-west-1", "eu-west-2", "eu-west-3", "eu-north-1", "sa-east-1"]') }}
45-
46-
outputs:
47-
layer-version: ${{ env.LAYER_VERSION }}
56+
region: ${{ inputs.region == 'ALL' && fromJson('["us-east-1", "us-east-2", "us-west-1", "us-west-2", "ap-south-1", "ap-northeast-1", "ap-northeast-2", "ap-northeast-3", "ap-southeast-1", "ap-southeast-2", "ca-central-1", "eu-central-1", "eu-west-1", "eu-west-2", "eu-west-3", "eu-north-1", "sa-east-1"]') || fromJson(format('["{0}"]', inputs.region)) }}
4857
permissions:
4958
id-token: write
50-
contents: read
51-
5259
steps:
53-
- name: Validate confirmation
60+
- name: Configure AWS credentials
61+
uses: aws-actions/configure-aws-credentials@v4
62+
with:
63+
role-to-assume: ${{ secrets.STRANDS_LAMBDA_LAYER_PUBLISHER_ROLE }}
64+
aws-region: ${{ matrix.region }}
65+
66+
- name: Create S3 bucket
5467
run: |
55-
CONFIRM="${{ inputs.confirm }}"
56-
if [ "$CONFIRM" != "Create Lambda Layer" ]; then
57-
if [[ "$CONFIRM" =~ ^(x86_64|aarch64|3\.[0-9]+|[a-z]+-[a-z]+-[0-9]+)$ ]]; then
58-
echo "Error: You entered '$CONFIRM' which looks like an architecture, Python version, or region."
59-
echo "Please type exactly 'Create Lambda Layer' to confirm."
68+
REGION="${{ matrix.region }}"
69+
ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)
70+
REGIONAL_BUCKET="${{ env.BUCKET_NAME }}-${ACCOUNT_ID}-${REGION}"
71+
72+
if ! aws s3api head-bucket --bucket "$REGIONAL_BUCKET" 2>/dev/null; then
73+
if [ "$REGION" = "us-east-1" ]; then
74+
aws s3api create-bucket --bucket "$REGIONAL_BUCKET" --region "$REGION" 2>/dev/null || echo "Bucket $REGIONAL_BUCKET already exists"
6075
else
61-
echo "Confirmation failed. You must type exactly 'Create Lambda Layer' to proceed."
76+
aws s3api create-bucket --bucket "$REGIONAL_BUCKET" --region "$REGION" --create-bucket-configuration LocationConstraint="$REGION" 2>/dev/null || echo "Bucket $REGIONAL_BUCKET already exists"
6277
fi
63-
exit 1
78+
echo "S3 bucket ready: $REGIONAL_BUCKET"
79+
else
80+
echo "S3 bucket already exists: $REGIONAL_BUCKET"
6481
fi
65-
echo "Confirmation validated"
6682
67-
- name: Checkout current repository
68-
uses: actions/checkout@v4
83+
package-and-upload:
84+
needs: create-buckets
85+
runs-on: ubuntu-latest
86+
strategy:
87+
matrix:
88+
python-version: ${{ inputs.python_version == 'ALL' && fromJson('["3.10", "3.11", "3.12", "3.13"]') || fromJson(format('["{0}"]', inputs.python_version)) }}
89+
architecture: ${{ inputs.architecture == 'ALL' && fromJson('["x86_64", "aarch64"]') || fromJson(format('["{0}"]', inputs.architecture)) }}
90+
region: ${{ inputs.region == 'ALL' && fromJson('["us-east-1", "us-east-2", "us-west-1", "us-west-2", "ap-south-1", "ap-northeast-1", "ap-northeast-2", "ap-northeast-3", "ap-southeast-1", "ap-southeast-2", "ca-central-1", "eu-central-1", "eu-west-1", "eu-west-2", "eu-west-3", "eu-north-1", "sa-east-1"]') || fromJson(format('["{0}"]', inputs.region)) }}
91+
92+
permissions:
93+
id-token: write
6994

95+
steps:
7096
- name: Set up Python
7197
uses: actions/setup-python@v4
7298
with:
@@ -94,34 +120,64 @@ jobs:
94120
run: |
95121
cd layer
96122
zip -r ../lambda-layer.zip .
97-
98-
- name: Upload layer to S3 and publish
123+
124+
- name: Upload to S3
99125
run: |
100126
PYTHON_VERSION="${{ matrix.python-version }}"
101127
ARCH="${{ matrix.architecture }}"
102128
REGION="${{ matrix.region }}"
103129
LAYER_NAME="strands-agents-py${PYTHON_VERSION//./_}-${ARCH}"
104-
BUCKET_NAME="strands-agents-lambda-layers-$(aws sts get-caller-identity --query Account --output text)-${REGION}"
130+
ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)
131+
BUCKET_NAME="${{ env.BUCKET_NAME }}-${ACCOUNT_ID}-${REGION}"
105132
LAYER_KEY="$LAYER_NAME/v${{ inputs.package_version }}/lambda-layer.zip"
106133
107-
if ! aws s3api head-bucket --bucket "$BUCKET_NAME" 2>/dev/null; then
108-
if [ "$REGION" = "us-east-1" ]; then
109-
aws s3api create-bucket --bucket "$BUCKET_NAME" --region "$REGION"
110-
else
111-
aws s3api create-bucket --bucket "$BUCKET_NAME" --region "$REGION" --create-bucket-configuration LocationConstraint="$REGION"
112-
fi
113-
fi
114-
115134
aws s3 cp lambda-layer.zip "s3://$BUCKET_NAME/$LAYER_KEY" --region "$REGION"
116135
echo "Uploaded layer to s3://$BUCKET_NAME/$LAYER_KEY"
117136
137+
publish-layer:
138+
needs: package-and-upload
139+
runs-on: ubuntu-latest
140+
strategy:
141+
matrix:
142+
python-version: ${{ inputs.python_version == 'ALL' && fromJson('["3.10", "3.11", "3.12", "3.13"]') || fromJson(format('["{0}"]', inputs.python_version)) }}
143+
architecture: ${{ inputs.architecture == 'ALL' && fromJson('["x86_64", "aarch64"]') || fromJson(format('["{0}"]', inputs.architecture)) }}
144+
region: ${{ inputs.region == 'ALL' && fromJson('["us-east-1", "us-east-2", "us-west-1", "us-west-2", "ap-south-1", "ap-northeast-1", "ap-northeast-2", "ap-northeast-3", "ap-southeast-1", "ap-southeast-2", "ca-central-1", "eu-central-1", "eu-west-1", "eu-west-2", "eu-west-3", "eu-north-1", "sa-east-1"]') || fromJson(format('["{0}"]', inputs.region)) }}
145+
146+
permissions:
147+
id-token: write
148+
149+
steps:
150+
- name: Configure AWS credentials
151+
uses: aws-actions/configure-aws-credentials@v4
152+
with:
153+
role-to-assume: ${{ secrets.STRANDS_LAMBDA_LAYER_PUBLISHER_ROLE }}
154+
aws-region: ${{ matrix.region }}
155+
156+
- name: Publish layer
157+
run: |
158+
PYTHON_VERSION="${{ matrix.python-version }}"
159+
ARCH="${{ matrix.architecture }}"
160+
REGION="${{ matrix.region }}"
161+
LAYER_NAME="strands-agents-py${PYTHON_VERSION//./_}-${ARCH}"
162+
ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)
163+
REGION_BUCKET="${{ env.BUCKET_NAME }}-${ACCOUNT_ID}-${REGION}"
164+
LAYER_KEY="$LAYER_NAME/v${{ inputs.package_version }}/lambda-layer.zip"
165+
118166
DESCRIPTION="PyPI package: strands-agents v${{ inputs.package_version }} (Python $PYTHON_VERSION, $ARCH)"
119167
168+
# Set compatible architecture based on matrix architecture
169+
if [ "$ARCH" = "x86_64" ]; then
170+
COMPATIBLE_ARCH="x86_64"
171+
else
172+
COMPATIBLE_ARCH="arm64"
173+
fi
174+
120175
LAYER_OUTPUT=$(aws lambda publish-layer-version \
121176
--layer-name $LAYER_NAME \
122177
--description "$DESCRIPTION" \
123-
--content S3Bucket=$BUCKET_NAME,S3Key=$LAYER_KEY \
178+
--content S3Bucket=$REGION_BUCKET,S3Key=$LAYER_KEY \
124179
--compatible-runtimes python${{ matrix.python-version }} \
180+
--compatible-architectures $COMPATIBLE_ARCH \
125181
--region "$REGION" \
126182
--license-info Apache-2.0 \
127183
--output json)
@@ -140,47 +196,3 @@ jobs:
140196
--region "$REGION"
141197
142198
echo "Successfully published layer version $LAYER_VERSION in region $REGION"
143-
144-
if [ "${{ env.IS_FULL_DEPLOY }}" = "true" ] && [ "$REGION" = "us-east-1" ] && [ "$PYTHON_VERSION" = "3.10" ] && [ "$ARCH" = "x86_64" ]; then
145-
echo "LAYER_VERSION=$LAYER_VERSION" >> $GITHUB_ENV
146-
fi
147-
148-
update-docs:
149-
if: ${{ env.IS_FULL_DEPLOY == 'true' }}
150-
needs: publish-layer
151-
runs-on: ubuntu-latest
152-
steps:
153-
- name: Checkout docs repository
154-
uses: actions/checkout@v4
155-
with:
156-
repository: ${{ github.repository_owner }}/docs
157-
token: ${{ secrets.GITHUB_TOKEN }}
158-
path: docs
159-
160-
- name: Update lambda layers documentation
161-
run: |
162-
cd docs
163-
LAYER_VERSION="${{ needs.publish-layer.outputs.layer-version }}"
164-
NEW_ROW="| $LAYER_VERSION | [${{ inputs.package_version }}](https://pypi.org/project/strands-agents/${{ inputs.package_version }}) | \`arn:aws:lambda:{REGION}:856699698935:layer:strands-agents-{VERSION}-{ARCHITECTURE}:$LAYER_VERSION\` |"
165-
166-
sed -i "/<!-- LAYER_TABLE_START -->/a\$NEW_ROW" docs/user-guide/deploy/lambda-layers.md
167-
168-
- name: Create Pull Request
169-
run: |
170-
cd docs
171-
git config user.name "github-actions[bot]"
172-
git config user.email "github-actions[bot]@users.noreply.github.com"
173-
174-
BRANCH="update-lambda-layers-${{ inputs.package_version }}"
175-
git checkout -b "$BRANCH"
176-
git add docs/user-guide/deploy/lambda-layers.md
177-
git commit -m "Update lambda layers with version ${{ inputs.package_version }}"
178-
git push origin "$BRANCH"
179-
180-
gh pr create \
181-
--title "Update lambda layers documentation for v${{ inputs.package_version }}" \
182-
--body "Automated update to add new lambda layer version ${{ inputs.package_version }}" \
183-
--head "$BRANCH" \
184-
--base main
185-
env:
186-
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

‎.github/workflows/yank-lambda-layer.yml‎

Lines changed: 81 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,81 @@
1+
name: Yank Lambda Layer
2+
3+
on:
4+
workflow_dispatch:
5+
inputs:
6+
layer_version:
7+
description: 'Layer version to yank'
8+
required: true
9+
type: string
10+
python_version:
11+
description: 'Python version'
12+
required: true
13+
default: 'ALL'
14+
type: choice
15+
options: ['ALL', '3.10', '3.11', '3.12', '3.13']
16+
architecture:
17+
description: 'Architecture'
18+
required: true
19+
default: 'ALL'
20+
type: choice
21+
options: ['ALL', 'x86_64', 'aarch64']
22+
region:
23+
description: 'AWS region'
24+
required: true
25+
default: 'ALL'
26+
type: choice
27+
# Only non opt-in regions included for now
28+
options: ['ALL', 'us-east-1', 'us-east-2', 'us-west-1', 'us-west-2', 'ap-south-1', 'ap-northeast-1', 'ap-northeast-2', 'ap-northeast-3', 'ap-southeast-1', 'ap-southeast-2', 'ca-central-1', 'eu-central-1', 'eu-west-1', 'eu-west-2', 'eu-west-3', 'eu-north-1', 'sa-east-1']
29+
confirm:
30+
description: 'Type "Yank Lambda Layer {layer version}" to confirm yanking the layer'
31+
required: true
32+
type: string
33+
34+
jobs:
35+
yank-layer:
36+
runs-on: ubuntu-latest
37+
continue-on-error: true
38+
strategy:
39+
fail-fast: false
40+
matrix:
41+
python-version: ${{ inputs.python_version == 'ALL' && fromJson('["3.10", "3.11", "3.12", "3.13"]') || fromJson(format('["{0}"]', inputs.python_version)) }}
42+
architecture: ${{ inputs.architecture == 'ALL' && fromJson('["x86_64", "aarch64"]') || fromJson(format('["{0}"]', inputs.architecture)) }}
43+
region: ${{ inputs.region == 'ALL' && fromJson('["us-east-1", "us-east-2", "us-west-1", "us-west-2", "ap-south-1", "ap-northeast-1", "ap-northeast-2", "ap-northeast-3", "ap-southeast-1", "ap-southeast-2", "ca-central-1", "eu-central-1", "eu-west-1", "eu-west-2", "eu-west-3", "eu-north-1", "sa-east-1"]') || fromJson(format('["{0}"]', inputs.region)) }}
44+
45+
permissions:
46+
id-token: write
47+
48+
steps:
49+
- name: Validate confirmation
50+
run: |
51+
CONFIRM="${{ inputs.confirm }}"
52+
EXPECTED="Yank Lambda Layer ${{ inputs.layer_version }}"
53+
if [ "$CONFIRM" != "$EXPECTED" ]; then
54+
echo "Confirmation failed. You must type exactly '$EXPECTED' to proceed."
55+
exit 1
56+
fi
57+
echo "Confirmation validated"
58+
59+
- name: Configure AWS credentials
60+
uses: aws-actions/configure-aws-credentials@v4
61+
with:
62+
role-to-assume: ${{ secrets.STRANDS_LAMBDA_LAYER_PUBLISHER_ROLE }}
63+
aws-region: ${{ matrix.region }}
64+
65+
- name: Yank layer
66+
run: |
67+
PYTHON_VERSION="${{ matrix.python-version }}"
68+
ARCH="${{ matrix.architecture }}"
69+
REGION="${{ matrix.region }}"
70+
LAYER_NAME="strands-agents-py${PYTHON_VERSION//./_}-${ARCH}"
71+
LAYER_VERSION="${{ inputs.layer_version }}"
72+
73+
echo "Attempting to yank layer $LAYER_NAME version $LAYER_VERSION in region $REGION"
74+
75+
# Delete the layer version completely
76+
aws lambda delete-layer-version \
77+
--layer-name $LAYER_NAME \
78+
--version-number $LAYER_VERSION \
79+
--region "$REGION"
80+
81+
echo "Completed yank attempt for layer $LAYER_NAME version $LAYER_VERSION in region $REGION"

0 commit comments

Comments
 (0)