feat: add GitHub Apps authentication support #459

hippolin · 2025-10-31T07:16:43Z

Summary

Add GitHub Apps authentication support to the Concourse Git resource, enabling users to authenticate with GitHub using GitHub Apps instead of personal access tokens. Includes comprehensive unit tests for the new functionality.

Changes

GitHub Apps Authentication: Implement setup_github_app_credentials() function to generate JWT tokens and retrieve installation access tokens
URI Handling: Automatic conversion of SSH URIs to HTTPS when using GitHub Apps authentication
Credential Management: Secure credential configuration for git operations
Dependencies: Add openssl and curl to Docker image for JWT generation and API calls
Schema: Add GitHub App configuration fields to source schema
Testing: Comprehensive unit tests covering all authentication scenarios

Files Changed

File	Changes
`Dockerfile`	Add openssl and curl dependencies
`assets/check`	Call setup_github_app_credentials
`assets/in`	Call setup_github_app_credentials
`assets/out`	Call setup_github_app_credentials
`assets/common.sh`	Add setup_github_app_credentials function
`assets/source_schema.json`	Add github_app_id, github_app_installation_id, github_app_private_key fields
`test/github_app.sh`	New comprehensive test suite (10 test cases)
`test/all.sh`	Include github_app.sh in test suite

Features

JWT token generation for GitHub Apps
Installation access token retrieval
Automatic SSH to HTTPS URI conversion
Secure credential configuration
Comprehensive error handling

Test Coverage

The new test suite (test/github_app.sh) includes:

Credential extraction and validation
Incomplete credential handling
Empty field edge cases
JSON parsing with missing fields
SSH URI format conversions
HTTPS URI preservation
Various authentication scenarios

All tests pass successfully:

$ bash test/github_app.sh
✓ All 10 tests passed

linux-foundation-easycla · 2025-10-31T07:16:48Z

The committers listed above are authorized under a signed CLA.

✅ login: hippolin / name: Hippo Lin (1b1c45c, 299e6a5, 845ea88, 8668a54, 9334e78, 9e765db)

Extract new variables from payload for GitHub Apps (app ID, private key, and installation ID). Generate JWT token and use it to fetch an access token via API. Configure Git credentials to use the token for repository access, enhancing authentication options in CI/CD workflows. Signed-off-by: Hippo Lin <hippo@hippo.idv.tw>

This introduces new options for GitHub App integration, enabling secure authentication via app ID, installation ID, and private key. These fields enhance support for automated Git operations in the source schema. Signed-off-by: Hippo Lin <hippo@hippo.idv.tw>

This update installs openssl and curl to ensure the image includes necessary tools for secure communications and HTTP requests, which may be required for enhanced functionality in the build environment. - Specifically, add these packages to the apk install command to support potential encryption and network operations. Signed-off-by: Hippo Lin <hippo@hippo.idv.tw>

Extract and modularize GitHub Apps token generation into a new setup_github_app_credentials function in common.sh for reuse. This change moves the logic from assets/check to assets/common.sh, and integrates it into assets/in and assets/out. It enhances modularity, reduces duplication, and enables secure authentication for Git operations using GitHub Apps. - Remove redundant debugging code in assets/check - Add function to handle JWT generation and credential setup Signed-off-by: Hippo Lin <hippo@hippo.idv.tw> (+1 squashed commit) Squashed commits: [bfe3a76] chore(check): add debug logging for payload This introduces a debug statement to write the incoming payload to /tmp/debug.json. This helps in troubleshooting issues by allowing easy inspection of the payload content during script execution. - The change is non-intrusive and only activates for debugging purposes. Signed-off-by: Hippo Lin <hippo@hippo.idv.tw>

…ction Add comprehensive unit tests for the new setup_github_app_credentials function that was added for GitHub Apps authentication support. Tests cover: - Skipping setup when credentials are not provided - Handling incomplete credentials - Credential extraction and parsing - SSH to HTTPS URI conversion - Empty field handling - JSON parsing edge cases 🤖 Generated with Claude Code Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Hippo Lin <hippo@hippo.idv.tw>

🤖 Generated with Claude Code Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Hippo Lin <hippo@hippo.idv.tw>

taylorsilva · 2025-10-31T15:49:55Z

Thanks for the PR! Gave this a quick glance and looks very comprehensive. I see tests added already. I've added this to my review backlog.

taylorsilva

Thanks for the PR and your patience in waiting for my review.

The tests don't test the code in setup_github_app_credentials() enough. You've copied bits of the function and tested them in isolation, but these tests don't actually guard against changes made to the function. Drift between the tests and the code in setup_github_app_credentials() is therefore possible.

I think with a few changes you could get the function fully tested. See individual comments below.

There is also the issue with openssl not supporting newer key formats. Not sure if there's an alternative way to generate the jwt signature that doesn't require openssl.

taylorsilva · 2025-11-24T19:31:29Z

test/github_app.sh

+    # Simulate the URI conversion logic
+    if [[ $uri == git@github.com:* ]]; then
+        uri="https://github.com/${uri#git@github.com:}"
+    fi


This doesn't actually test the code that gets used in common.sh.

Why not put the code from common.sh into a function and then test the function?

taylorsilva · 2025-11-24T19:51:18Z

test/github_app.sh

+    # Extract the values to verify they are extracted correctly
+    github_app_id=$(jq -r '.source.github_app_id // ""' <<< "$payload")
+    github_app_private_key=$(jq -r '.source.github_app_private_key // ""' <<< "$payload")
+    github_app_installation_id=$(jq -r '.source.github_app_installation_id // ""' <<< "$payload")


This is also not testing the code in common.sh Why not move it out into a function so it can be tested?

You could export all of these vars and then call setup_github_app_credentials.

taylorsilva · 2025-11-24T20:43:45Z

assets/source_schema.json

+  "github_app_id": "",
+  "github_app_installation_id": "",
+  "github_app_private_key": ""


These new fields should be documented in the README please.

taylorsilva · 2025-11-24T20:53:41Z

assets/common.sh

+    exp=$((now + 600))  # 10 minutes expiration
+    jwt_header=$(echo -n '{"alg":"RS256","typ":"JWT"}' | base64 | tr -d '=' | tr '/+' '_-' | tr -d '\r\n')
+    jwt_payload=$(echo -n "{\"iat\":$iat,\"exp\":$exp,\"iss\":\"$github_app_id\"}" | base64 | tr -d '=' | tr '/+' '_-' | tr -d '\r\n')
+    jwt_signature=$(echo -n "$jwt_header.$jwt_payload" | openssl dgst -sha256 -sign <(echo -n "$github_app_private_key") | base64 | tr -d '=' | tr '/+' '_-' | tr -d '\r\n')


openssl only supports private keys in PEM format. If the key is not in PEM format you get this error:

Could not find private key from /dev/fd/63 20602F81FFFF0000:error:1608010C:STORE routines:ossl_store_handle_load_result:unsupported:crypto/store/store_result.c:162:provider=default

PEM formatted keys start with:

-----BEGIN RSA PRIVATE KEY-----

Newer openssh keys, which openssl cannot read, start with:

-----BEGIN OPENSSH PRIVATE KEY-----

This should be documented in the README and maybe even detected by the function here and errored out to the user.

I did some digging to see if it's possible to convert ED25519 keys (the default from ssh-keygen) to PEM format and couldn't turn anything up indicating that it's possible.

taylorsilva · 2025-11-24T20:55:45Z

test/github_app.sh

+    uri="git@github.com:user/repo.git"
+    if [[ $uri == git@github.com:* ]]; then
+        uri="https://github.com/${uri#git@github.com:}"
+    fi
+    test "$uri" = "https://github.com/user/repo.git"
+
+    # Test without .git extension
+    uri="git@github.com:user/repo"
+    if [[ $uri == git@github.com:* ]]; then
+        uri="https://github.com/${uri#git@github.com:}"
+    fi
+    test "$uri" = "https://github.com/user/repo"
+}
+
+# Test case 10: Verify HTTPS URIs are not modified
+it_does_not_modify_https_uris() {
+    uri="https://github.com/test/repo.git"
+
+    # Should not modify HTTPS URIs
+    if [[ $uri == git@github.com:* ]]; then
+        uri="https://github.com/${uri#git@github.com:}"
+    fi


Same as previous comment from it_converts_ssh_uri_to_https()

taylorsilva · 2025-11-24T21:09:13Z

assets/common.sh

+    access_token=$(curl -s -X POST -H "Authorization: Bearer $jwt_token" -H "Accept: application/vnd.github.v3+json" \
+      "https://api.github.com/app/installations/$github_app_installation_id/access_tokens" | jq -r '.token')


I'm guessing this curl call is what stopped you from directly testing this function and resulted in the copy-pasted tests.

We can do two things here to actually test this curl call:

Make the api endpoint https://api.github.com configurable

Use netcat nc to mock the server response

Basic example of using netcat to mock a server response:

echo -e "HTTP/1.1 200 OK\r\nContent-Type: application/json\r\n\r\n{\"token\":\"some-token\"}" | nc -l 8080 & curl http://localhost:8080

Another idea! In the test suite, you could add a curl() function that gets called instead of real curl.

curl() { // test and check passed in args } export -f curl

There are options, we can get this code properly tested :)

taylorsilva · 2025-11-24T21:10:31Z

test/github_app.sh

+
+# Test case 3: setup_github_app_credentials extracts credentials correctly
+it_extracts_github_app_credentials() {
+    local test_key="test-private-key-content"


Can use ssh-keygen to generate keys. We do this in the check tests:

git-resource/test/check.sh

Lines 101 to 103 in 6e7bbd3

local key=$TMPDIR/key-no-passphrase

ssh-keygen -f $key

These keys end up in ED25519 openssh format though, which I mentioned openssl won't be able to read.

You can use jq -n --arg key "$key" to have jq properly escape the key in the json payload.

jq -n --arg key "$key" '{ "source": { "uri": "https://github.com/test/repo.git", "github_app_id": "123456", "github_app_private_key": $key, "github_app_installation_id": "789012" } }'

hippolin requested a review from a team as a code owner October 31, 2025 07:16

hippolin and others added 6 commits October 31, 2025 15:27

chore(test): make github_app.sh executable

8668a54

🤖 Generated with Claude Code Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Hippo Lin <hippo@hippo.idv.tw>

hippolin force-pushed the feature/github-apps-auth-support branch from 999af8c to 8668a54 Compare October 31, 2025 07:30

taylorsilva added this to Pull Requests Oct 31, 2025

github-project-automation bot moved this to Todo in Pull Requests Oct 31, 2025

taylorsilva moved this from Todo to In Progress in Pull Requests Nov 24, 2025

taylorsilva requested changes Nov 24, 2025

View reviewed changes

taylorsilva moved this from In Progress to Waiting on Contributor in Pull Requests Nov 24, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

feat: add GitHub Apps authentication support #459

feat: add GitHub Apps authentication support #459

Uh oh!

hippolin commented Oct 31, 2025

Uh oh!

linux-foundation-easycla bot commented Oct 31, 2025 •

edited

Loading

Uh oh!

taylorsilva commented Oct 31, 2025

Uh oh!

taylorsilva left a comment •

edited

Loading

Uh oh!

taylorsilva Nov 24, 2025

Uh oh!

taylorsilva Nov 24, 2025

Uh oh!

taylorsilva Nov 24, 2025

Uh oh!

taylorsilva Nov 24, 2025

Uh oh!

taylorsilva Nov 24, 2025

Uh oh!

taylorsilva Nov 24, 2025

Uh oh!

taylorsilva Nov 24, 2025

Uh oh!

taylorsilva Nov 24, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

		access_token=$(curl -s -X POST -H "Authorization: Bearer $jwt_token" -H "Accept: application/vnd.github.v3+json" \
		"https://api.github.com/app/installations/$github_app_installation_id/access_tokens" \| jq -r '.token')

Uh oh!

feat: add GitHub Apps authentication support #459

Are you sure you want to change the base?

feat: add GitHub Apps authentication support #459

Uh oh!

Conversation

hippolin commented Oct 31, 2025

Summary

Changes

Files Changed

Features

Test Coverage

Uh oh!

linux-foundation-easycla bot commented Oct 31, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

taylorsilva commented Oct 31, 2025

Uh oh!

taylorsilva left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

linux-foundation-easycla bot commented Oct 31, 2025 •

edited

Loading

taylorsilva left a comment •

edited

Loading