Skip to content

Commit e3eb3bd

Browse files
Copilotchagong
Copilot
and
chagong
authored
Add AI Triage Workflow and LLM Guidelines for Automated Issue Management (#1789)
* Initial plan * Add AI triage workflow and LLM guidelines for vscode-java-test Co-authored-by: chagong <831821+chagong@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: chagong <831821+chagong@users.noreply.github.com>
1 parent a9883c2 commit e3eb3bd

File tree

2 files changed

+171
-0
lines changed

2 files changed

+171
-0
lines changed

‎.github/llms.md‎

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
# Test Runner for Java
2+
3+
Test Runner for Java is a lightweight extension to run and debug Java test cases in Visual Studio Code. It works with Language Support for Java by Red Hat to provide comprehensive Java testing capabilities.
4+
5+
## Features
6+
- Run & Debug JUnit/TestNG Test Cases
7+
- Test discovery and execution
8+
- Test result visualization
9+
- Integration with Java project structure
10+
- Support for Maven and Gradle projects
11+
12+
## Label
13+
When labeling an issue, follow the rules below per label category:
14+
### General Rules
15+
- Analyze if the issue is related with the scope of Java testing functionality in VS Code. If not, STOP labelling IMMEDIATELY.
16+
- Assign label per category.
17+
- If a category is not applicable or you're unsure, you may skip it.
18+
- Do not assign multiple labels within the same category, unless explicitly allowed as an exception.
19+
20+
### Issue Type Labels
21+
- [bug]: Primary label for real bug issues
22+
- [enhancement]: Primary label for enhancement issues
23+
- [documentation]: Primary label for documentation issues
24+
- [question]: Primary label for question issues
25+
26+
### Test Framework Labels
27+
- [junit]: Issues specific to JUnit test framework
28+
- [testng]: Issues specific to TestNG test framework
29+
30+
### Project Type Labels
31+
- [maven]: Issues related to Maven projects
32+
- [gradle]: Issues related to Gradle projects
33+
34+
### Component Labels
35+
- [test-discovery]: Issues related to finding and discovering tests
36+
- [test-execution]: Issues related to running tests
37+
- [test-debugging]: Issues related to debugging tests
38+
- [test-reporting]: Issues related to test results and reporting
39+
- [ui]: Issues related to user interface and experience
40+
41+
### Priority Labels
42+
- [high-priority]: Critical issues that significantly impact functionality
43+
- [low-priority]: Nice-to-have improvements or minor issues
44+
45+
### Status Labels
46+
- [waiting-for-user-info]: Waiting for additional information from the user
47+
- [investigating]: Issue is being investigated
48+
- [needs-reproduction]: Issue needs to be reproduced
49+
- [fixed-next-release]: Issue has been fixed and will be in the next release

‎.github/workflows/triage-agent.yml‎

Lines changed: 122 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,122 @@
1+
name: AI Triage - Label and Comment on New Issues
2+
on:
3+
issues:
4+
types: [opened]
5+
workflow_dispatch:
6+
inputs:
7+
issue_number:
8+
description: 'Issue number to triage (manual run). e.g. 123'
9+
required: true
10+
11+
permissions:
12+
issues: write
13+
contents: read
14+
15+
jobs:
16+
label_and_comment:
17+
runs-on: ubuntu-latest
18+
19+
steps:
20+
- name: Checkout repository
21+
uses: actions/checkout@v2
22+
23+
- name: Get issue data
24+
id: get_issue
25+
uses: actions/github-script@v6
26+
with:
27+
script: |
28+
const eventName = context.eventName;
29+
let issue;
30+
if (eventName === 'workflow_dispatch') {
31+
const inputs = context.payload.inputs || {};
32+
const issueNumber = inputs.issue_number || inputs.issueNumber;
33+
if (!issueNumber) core.setFailed('Input issue_number is required for manual run.');
34+
const { data } = await github.rest.issues.get({
35+
owner: context.repo.owner,
36+
repo: context.repo.repo,
37+
issue_number: parseInt(issueNumber, 10),
38+
});
39+
issue = data;
40+
} else if (context.payload.issue) {
41+
issue = context.payload.issue;
42+
} else {
43+
core.setFailed('No issue information found in the event payload.');
44+
}
45+
core.setOutput('id', String(issue.number));
46+
core.setOutput('user', String((issue.user && issue.user.login) || ''));
47+
core.setOutput('title', String(issue.title || ''));
48+
core.setOutput('body', String(issue.body || ''));
49+
const labelNames = (issue.labels || []).map(label => label.name);
50+
core.setOutput('labels', JSON.stringify(labelNames));
51+
52+
- name: Call Azure Function
53+
id: call_azure_function
54+
env:
55+
PAYLOAD: >-
56+
{
57+
"authToken": "${{ secrets.GITHUB_TOKEN }}",
58+
"repoId": "microsoft/vscode-java-test",
59+
"issueData": {
60+
"id": ${{ steps.get_issue.outputs.id }},
61+
"user": ${{ toJson(steps.get_issue.outputs.user) }},
62+
"title": ${{ toJson(steps.get_issue.outputs.title) }},
63+
"body": ${{ toJson(steps.get_issue.outputs.body) }},
64+
"labels": ${{ steps.get_issue.outputs.labels }}
65+
},
66+
"mode": "DirectUpdate"
67+
}
68+
69+
run: |
70+
# Make the HTTP request with improved error handling and timeouts
71+
echo "Making request to triage agent..."
72+
73+
# Add timeout handling and better error detection
74+
set +e # Don't exit on curl failure
75+
response=$(timeout ${{ vars.TRIAGE_AGENT_TIMEOUT }} curl \
76+
--max-time 0 \
77+
--connect-timeout 30 \
78+
--fail-with-body \
79+
--silent \
80+
--show-error \
81+
--write-out "HTTPSTATUS:%{http_code}" \
82+
--header "Content-Type: application/json" \
83+
--request POST \
84+
--data "$PAYLOAD" \
85+
${{ secrets.TRIAGE_FUNCTION_LINK }} 2>&1)
86+
87+
curl_exit_code=$?
88+
set -e # Re-enable exit on error
89+
90+
echo "Curl exit code: $curl_exit_code"
91+
92+
# Check if curl command timed out or failed
93+
if [ $curl_exit_code -eq 124 ]; then
94+
echo "❌ Request timed out after 650 seconds"
95+
exit 1
96+
elif [ $curl_exit_code -ne 0 ]; then
97+
echo "❌ Curl command failed with exit code: $curl_exit_code"
98+
echo "Response: $response"
99+
exit 1
100+
fi
101+
102+
# Extract HTTP status code and response body
103+
http_code=$(echo "$response" | grep -o "HTTPSTATUS:[0-9]*" | cut -d: -f2)
104+
response_body=$(echo "$response" | sed 's/HTTPSTATUS:[0-9]*$//')
105+
106+
echo "HTTP Status Code: $http_code"
107+
108+
# Validate HTTP status code
109+
if [ -z "$http_code" ]; then
110+
echo "❌ Failed to extract HTTP status code from response"
111+
echo "Raw response: $response"
112+
exit 1
113+
fi
114+
115+
# Check if the request was successful
116+
if [ "$http_code" -ge 200 ] && [ "$http_code" -lt 300 ]; then
117+
echo "✅ Azure Function call succeeded"
118+
else
119+
echo "❌ Azure Function call failed with status code: $http_code"
120+
echo "Response: $response_body"
121+
exit 1
122+
fi

0 commit comments

Comments
 (0)