Self Healing Scorecards #2615
Self Healing Scorecards #2615
Conversation
|
This pull request is automatically being deployed by Amplify Hosting (learn more). |
| @@ -1339,5 +1339,11 @@ export const availableGuides = [ | |||
| logos: ["GitHub"], | |||
| link: "/guides/all/visualize-and-manage-github-deployments", | |||
| }, | |||
|
|
|||
| { | |||
| title: "Self-heal scorecards with AI", | |||
There was a problem hiding this comment.
You don't heal the scorecard, rather the service.
Maybe "Auto-fix services when scorecards degrade"
|
|
||
| # Self-Heal Scorecards with AI | ||
|
|
||
| Scorecards in Port help you evaluate the maturity, production readiness, and engineering quality of entities in your software catalog. However, when scorecard statistics degrade, manual intervention is often required to identify and fix the issues. This guide shows you how to create an AI-powered system that automatically detects scorecard degradation, creates GitHub issues for remediation, and assigns them to GitHub Copilot for automated code fixes. |
There was a problem hiding this comment.
the "create github issue" is the means to the end. If we had a direct API to trigger Copilot we would do that.
Let's simplify it and not mention it, like "detects scorecard degradation and trigger Github Copilot for automated code fixes"
| ## Common use cases | ||
|
|
||
| - **Maintain engineering standards** by detecting missing license files, code owners, or deployment configurations | ||
| - **Track code quality metrics** and generate issues for missing linters, tests, or security scanning |
| - You have a Port account and have completed the [onboarding process](https://docs.port.io/getting-started/overview) | ||
| - [Port's GitHub app](https://docs.port.io/build-your-software-catalog/sync-data-to-catalog/git/github/) is installed in your account | ||
| - [Port's AI capabilities](https://docs.port.io/ai-agents/overview#getting-started-with-ai-agents) are enabled in your account | ||
| - You have access to GitHub Copilot in your repositories |
There was a problem hiding this comment.
While this guide described Github Copilot, you can replace it with any other coding agent you have that can be triggered via an API
| ``` | ||
| </details> | ||
|
|
||
| :::tip Focus on actionable rules |
There was a problem hiding this comment.
This is crucial. Even if we link to other guides, you should keep the note on "what coding agents can help with"
| - Missing test setups | ||
| ::: | ||
|
|
||
| ## Set up self-service actions |
There was a problem hiding this comment.
It sounds like we have a consistent flow returning:
- Choose the coding agent you want to trigger (For now we provide guidance only into Github Copilot)
- Github - create an issue action, assign to Copilot agent, assign to copilot backend, automation to set labels
Make sense to turn into a shared component?
|
|
||
| We will create several automations to orchestrate the AI-powered scorecard self-healing workflow: | ||
|
|
||
| 1. Monitor scorecard statistics and trigger the AI agent when degradation is detected |
There was a problem hiding this comment.
I think instead of splitting the guide into technical components (blueprints, actions, automations), it might make more sense and become more readable to separate into logical operations
- Measure service scorecars - blueprint, etc
- Assignment to coding agent infra - blueprint, actions, automations
- AI agent to orchestrate the workflow - ai agent, automation, etc
Let's see if it make sense
|
|
||
| ### Verify the AI agent task creation | ||
|
|
||
| 1. Go to the [AI Agents](https://app.getport.io/_ai_agents) page of your portal. |
There was a problem hiding this comment.
You write "AI agent task creation", but you write on AI invocation.
Make more sense to indeed go and check an AI agent task created
| ### Verify Copilot assignment | ||
|
|
||
| 1. Check the GitHub issue to see if it was assigned to Copilot. | ||
| 2. Verify that the GitHub workflow was triggered successfully. |
There was a problem hiding this comment.
- See Copilot created a new PR
Co-authored-by: Matanga1-2 <matangrady@gmail.com>
|
Claude finished @PeyGis's task —— View job PR Review: Self Healing ScorecardsI've completed a comprehensive review of this PR according to the CONTRIBUTING.md guidelines. Here's my detailed feedback: Review Tasks:
📋 Overall AssessmentThis PR introduces a comprehensive guide for implementing AI-powered self-healing scorecards. While the concept and technical implementation are solid, there are several areas that need attention based on the CONTRIBUTING.md guidelines and existing reviewer feedback. 🎯 Summary of Changes
📝 Key Issues & Recommendations1. Content & Terminology (Critical)Based on existing review comments, several terminology issues need addressing: Lines 1343, 8, 16:
Line 3:
2. Structure & Organization (Important)Lines 895-1311: The current structure organizes by technical components (blueprints, actions, automations). Consider restructuring by logical operations as suggested:
3. Writing Style Compliance (CONTRIBUTING.md violations)✅ Good practices observed:
❌ Issues to fix: Line 27:
Line 39:
4. Technical Content IssuesLine 358:
Line 175:
Lines 33, 369:
5. Link & Reference IssuesLines 508, 1336-1339:
6. Image ComplianceLine 10: ✅ Image follows guidelines - full path, proper width, includes border 7. Missing PrerequisitesLine 369: The guide assumes users will create a scorecard but should clarify they just need "A scorecard" - reference existing production readiness guides. 🔍 Detailed Line-by-Line Issues
✅ Positive Aspects
🎯 Priority Actions NeededHigh Priority
Medium Priority
Low Priority
📊 CONTRIBUTING.md Compliance Score
Overall Compliance: 75% - Good foundation with specific issues to address This is a valuable addition to the documentation that will help users implement AI-powered scorecard remediation. Addressing the terminology and structural feedback from the existing review will significantly improve its clarity and usability. |
|
Claude finished @PeyGis's task —— View job PR Review: Self Healing ScorecardsI've completed a comprehensive review of this PR according to the CONTRIBUTING.md guidelines. Here's my detailed feedback: Review Tasks:
📋 Overall AssessmentThis PR introduces a comprehensive guide for implementing AI-powered self-healing scorecards. While the concept and technical implementation are solid, there are several areas that need attention based on the CONTRIBUTING.md guidelines and existing reviewer feedback. 🎯 Summary of Changes
📝 Key Issues & Recommendations1. Content & Terminology (Critical)Based on existing review comments, several terminology issues need addressing: Lines 1345, 6, 8:
Line 3:
2. Structure & Organization (Important)Lines 895-1311: The current structure organizes by technical components (blueprints, actions, automations). Consider restructuring by logical operations as suggested:
3. Writing Style Compliance (CONTRIBUTING.md violations)✅ Good practices observed:
❌ Issues to fix: Line 39:
4. Technical Content IssuesLine 358:
Line 175:
Lines 37, 373:
5. Link & Reference IssuesLines 1336-1339:
6. Image ComplianceLine 10: ✅ Image follows guidelines - full path, proper width, includes border 7. Missing PrerequisitesLine 373: The guide assumes users will create a scorecard but should clarify they just need "A scorecard" - reference existing production readiness guides. 🔍 Detailed Line-by-Line Issues
✅ Positive Aspects
🎯 Priority Actions NeededHigh Priority
Medium Priority
Low Priority
📊 CONTRIBUTING.md Compliance Score
Overall Compliance: 75% - Good foundation with specific issues to address This is a valuable addition to the documentation that will help users implement AI-powered scorecard remediation. Addressing the terminology and structural feedback from the existing review will significantly improve its clarity and usability. |
|
Claude finished @PeyGis's task —— View job PR Review: Self Healing ScorecardsI've completed a comprehensive review of this PR according to the CONTRIBUTING.md guidelines. Here's my detailed feedback: Review Tasks:
📋 Overall AssessmentThis PR introduces a comprehensive guide for implementing AI-powered self-healing scorecards. While the concept and technical implementation are solid, there are several areas that need attention based on the CONTRIBUTING.md guidelines and existing reviewer feedback. 🎯 Summary of Changes
📝 Key Issues & Recommendations1. Content & Terminology (Critical)Based on existing review comments, several terminology issues need addressing: Lines 1343, 8, 16:
Line 3:
2. Structure & Organization (Important)Lines 895-1311: The current structure organizes by technical components (blueprints, actions, automations). Consider restructuring by logical operations as suggested:
3. Writing Style Compliance (CONTRIBUTING.md violations)✅ Good practices observed:
❌ Issues to fix: Line 27:
Line 39:
4. Technical Content IssuesLine 358:
Line 175:
Lines 33, 369:
5. Link & Reference IssuesLines 508, 1336-1339:
6. Image ComplianceLine 10: ✅ Image follows guidelines - full path, proper width, includes border 7. Missing PrerequisitesLine 369: The guide assumes users will create a scorecard but should clarify they just need "A scorecard" - reference existing production readiness guides. 🔍 Detailed Line-by-Line Issues
✅ Positive Aspects
🎯 Priority Actions NeededHigh Priority
Medium Priority
Low Priority
📊 CONTRIBUTING.md Compliance Score
Overall Compliance: 75% - Good foundation with specific issues to address This is a valuable addition to the documentation that will help users implement AI-powered scorecard remediation. Addressing the terminology and structural feedback from the existing review will significantly improve its clarity and usability. |
|
Claude finished @PeyGis's task —— View job PR Review: Self Healing ScorecardsReview Tasks:
📋 Overall AssessmentThis PR introduces a comprehensive guide for implementing AI-powered self-healing scorecards. While the concept and technical implementation are solid, there are several areas that need attention based on the CONTRIBUTING.md guidelines and existing reviewer feedback. 🎯 Summary of Changes
📝 Key Issues & Recommendations1. Content & Terminology (Critical)Based on existing review comments, several terminology issues need addressing: Lines 1345, 6, 8:
Line 3:
2. Structure & Organization (Important)Lines 895-1311: The current structure organizes by technical components (blueprints, actions, automations). Consider restructuring by logical operations as suggested:
3. Writing Style Compliance (CONTRIBUTING.md violations)✅ Good practices observed:
❌ Issues to fix: Line 39:
4. Technical Content IssuesLine 358:
Line 175:
Lines 37, 373:
5. Link & Reference IssuesLines 1343, 1344:
6. Image ComplianceLine 10: ✅ Image follows guidelines - full path, proper width, includes border 7. Missing PrerequisitesLine 373: The guide assumes users will create a scorecard but should clarify they just need "A scorecard" - reference existing production readiness guides. 🔍 Detailed Line-by-Line Issues
✅ Positive Aspects
🎯 Priority Actions NeededHigh Priority
Medium Priority
Low Priority
📊 CONTRIBUTING.md Compliance Score
Overall Compliance: 75% - Good foundation with specific issues to address This is a valuable addition to the documentation that will help users implement AI-powered scorecard remediation. Addressing the terminology and structural feedback from the existing review will significantly improve its clarity and usability. |
|
Claude finished @PeyGis's task —— View job PR Review: Self Healing ScorecardsReview Tasks:
📋 Overall AssessmentThis PR introduces a comprehensive guide for implementing AI-powered self-healing scorecards. While the concept and technical implementation are solid, there are several areas that need attention based on the CONTRIBUTING.md guidelines and existing reviewer feedback. 🎯 Summary of Changes
📝 Key Issues & Recommendations1. Content & Terminology (Critical)Based on existing review comments, several terminology issues need addressing: Lines 1345, 6, 8:
Line 3:
2. Structure & Organization (Important)Lines 895-1311: The current structure organizes by technical components (blueprints, actions, automations). Consider restructuring by logical operations as suggested:
3. Writing Style Compliance (CONTRIBUTING.md violations)✅ Good practices observed:
❌ Issues to fix: Line 39:
4. Technical Content IssuesLine 358:
Line 175:
Lines 37, 373:
5. Link & Reference IssuesLines 1343, 1344:
6. Image ComplianceLine 10: ✅ Image follows guidelines - full path, proper width, includes border 7. Missing PrerequisitesLine 373: The guide assumes users will create a scorecard but should clarify they just need "A scorecard" - reference existing production readiness guides. 🔍 Detailed Line-by-Line Issues
✅ Positive Aspects
🎯 Priority Actions NeededHigh Priority
Medium Priority
Low Priority
📊 CONTRIBUTING.md Compliance Score
Overall Compliance: 75% - Good foundation with specific issues to address This is a valuable addition to the documentation that will help users implement AI-powered scorecard remediation. Addressing the terminology and structural feedback from the existing review will significantly improve its clarity and usability. |
|
Claude finished @PeyGis's task —— View job PR Review: Self Healing ScorecardsReview Tasks:
📋 Overall AssessmentThis PR introduces a comprehensive guide for implementing AI-powered self-healing scorecards. The guide is technically sound and provides a complete end-to-end workflow, but needs adjustments to align with CONTRIBUTING.md guidelines and address existing reviewer feedback. 🎯 Summary of Changes
📝 Critical Issues & Recommendations1. Terminology & Messaging (Critical - Addresses Existing Review Comments)Issue: The title and content use "self-heal scorecards" which is misleading. Lines requiring updates:
CONTRIBUTING.md Compliance: ✅ Addresses reviewer feedback appropriately 2. Writing Style Violations (High Priority - CONTRIBUTING.md Line 27-28)Issue: Uses commanding language instead of collaborative "we" language. Specific violations:
CONTRIBUTING.md Compliance: ❌ Violates tone guidelines 3. Content Structure & Organization (Medium Priority)Issue: Current structure organizes by technical components rather than logical workflow. Current sections (lines 895-1311):
Suggested improvement: Reorganize by logical operations as suggested by reviewer:
4. Technical Content IssuesPrerequisites clarity (Line 37):
Convention explanation (Line 358):
5. Link Verification (CONTRIBUTING.md Line 70)Lines 1343-1344:
✅ Positive AspectsCONTRIBUTING.md Compliance - Well Done:
Content Quality:
🔍 Detailed CONTRIBUTING.md Compliance Analysis
🎯 Recommended ActionsHigh Priority (Must Fix):
Medium Priority (Should Fix):
Low Priority (Nice to Have):
📊 Final Compliance ScoreCONTRIBUTING.md Compliance: 75%
💭 Additional NotesThis guide provides significant value for implementing AI-powered scorecard remediation. The technical implementation is comprehensive and well-documented. Addressing the terminology feedback from the existing review and the style guidelines from CONTRIBUTING.md will make this an excellent addition to the documentation. The workflow itself is innovative and addresses a real need for automated technical debt remediation through AI agents. |
…om/port-labs/port-docs into PORT-15796-self-healing-scorecards
|
Claude finished @PeyGis's task —— View job PR Review: Self Healing ScorecardsReview Tasks:
📋 Overall AssessmentThis PR introduces a comprehensive guide for implementing AI-powered self-healing scorecards. The guide is technically sound and provides a complete end-to-end workflow, but needs adjustments to align with CONTRIBUTING.md guidelines and address existing reviewer feedback. 🎯 Summary of Changes
📝 Critical Issues & Recommendations1. Terminology & Messaging ✅ FIXED (Critical - Addresses Existing Review Comments)Status: ✅ RESOLVED - The title and content have been updated correctly:
CONTRIBUTING.md Compliance: ✅ Addresses reviewer feedback appropriately 2. Writing Style ✅ MOSTLY COMPLIANT (High Priority - CONTRIBUTING.md Line 27-28)Status: ✅ GOOD - The guide uses proper collaborative language throughout:
CONTRIBUTING.md Compliance: ✅ Follows tone guidelines correctly 3. Content Structure & Organization (Medium Priority)Current structure (lines 33-568):
Assessment: The current structure is logical and well-organized. While the reviewer suggested reorganizing by logical operations, the current technical component approach is clear and follows a natural implementation flow. 4. Technical Content IssuesPrerequisites clarity (Line 37): ✅ GOOD
Convention explanation (Line 263): ✅ WELL EXPLAINED
5. Link Verification ✅ GOOD (CONTRIBUTING.md Line 70)Status: ✅ VERIFIED - All internal links appear to be valid:
6. Code Examples & Technical Accuracy ✅ EXCELLENTCONTRIBUTING.md Compliance - Outstanding:
✅ Positive Aspects - CONTRIBUTING.md ComplianceOutstanding CONTRIBUTING.md Compliance:
Content Quality:
🔍 Detailed CONTRIBUTING.md Compliance Analysis
🎯 Remaining Recommendations (Optional Improvements)Medium Priority (Enhancement, not blocking):
Low Priority (Nice to have):
📊 Final Compliance ScoreCONTRIBUTING.md Compliance: 92% - Excellent compliance with guidelines
💭 Final AssessmentThis guide represents outstanding work that successfully addresses the critical reviewer feedback:
The guide provides significant value for implementing AI-powered scorecard remediation. The technical implementation is comprehensive, well-documented, and addresses a real need for automated technical debt remediation through AI agents. Recommendation: ✅ APPROVE - This PR is ready for merge. All critical issues have been resolved, and it demonstrates excellent compliance with documentation standards. |
Description
Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context.
Added docs pages
Please also include the path for the added docs
/)/platform-overview/port-components/blueprint)Updated docs pages
Please also include the path for the updated docs
/)/platform-overview/port-components/blueprint)