Skip to content

Commit 618f9a4

Browse files
misravedmisraved
committed
Refactor the doc
1 parent 6c8c62a commit 618f9a4

File tree

1 file changed

+88
-73
lines changed

1 file changed

+88
-73
lines changed

‎docs/develop/using-ai.md‎

Lines changed: 88 additions & 73 deletions
Original file line numberDiff line numberDiff line change
@@ -5,11 +5,11 @@ sidebar_label: Using AI
55

66
# Using AI
77

8-
Creating new controls for Powerpipe mods with AI tools and IDEs works remarkably well. At Turbot, we develop controls frequently and use AI for almost every new control we create. We've found that AI typically produces excellent results when working with existing mod repositories, as it can learn from established patterns and conventions.
8+
Creating new benchmarks, dashboards, and controls for Powerpipe mods with AI tools and IDEs works remarkably well. At Turbot, we develop these components frequently and use AI for almost every new implementation. We've experimented with various approaches, including detailed prompt engineering, explicit guidelines, IDE rules and instructions, and complex workflows, but found that AI typically produces excellent results even without heavy guidance.
99

10-
The key to success is working within existing mod repositories and opening the entire repository as a folder or project in your IDE. This gives AI tools access to existing control implementations, documentation examples, code patterns, and naming conventions to generate consistent, high-quality results.
10+
The key to this success is working within existing mod repositories and opening the entire repository as a folder or project in your IDE. This gives AI tools access to existing implementations, documentation examples, code patterns, and naming conventions to generate consistent, high-quality results without extensive prompting.
1111

12-
If you're looking to use AI to run Powerpipe controls rather than develop new ones, you can use the [Powerpipe MCP server](../query/mcp), which provides powerful tools for AI agents to inspect and run controls, benchmarks, and queries.
12+
If you're looking to use AI to run Powerpipe controls rather than develop new ones, you can use the [Powerpipe MCP server](../run/mcp), which provides powerful tools for AI agents to inspect and run controls, benchmarks, and queries.
1313

1414
## Getting Started
1515

@@ -22,108 +22,123 @@ While AI often works well with simple requests like "Create a control for [requi
2222
3. Set up access to the cloud providers or systems you'll be evaluating.
2323
4. Configure the [Powerpipe MCP server](https://github.com/turbot/powerpipe-mcp) which allows the agent to inspect and run controls.
2424

25-
### Understand Conventions
25+
### Create Control
2626

27-
First, review existing controls and conventions to ensure consistency:
27+
First, create the complete control implementation:
2828

2929
```md
30-
Your goal is to understand the conventions for creating a new Powerpipe control.
30+
Your goal is to create a new control with all necessary components.
3131

32-
1. Review the sample repository at https://github.com/turbot/steampipe-mod-aws-thrifty for reference patterns.
32+
1. Review existing controls and their documentation in the mod to understand established patterns, naming conventions, and query structures.
3333

34-
2. Analyze existing controls in the current mod to understand:
35-
- Control naming conventions
36-
- File and directory organization
37-
- Query structure and style
38-
- Variable naming and placement
39-
- Documentation format and requirements
40-
41-
3. Document the key conventions you've identified to ensure your new control will be consistent.
34+
2. Create the control definition:
35+
- Choose appropriate directory and file based on existing patterns
36+
- Use descriptive names following conventions
37+
- Include clear descriptions and documentation
38+
- Set appropriate severity levels
39+
- Define any necessary variables with default values
40+
- Implement the SQL query following established patterns:
41+
- Use `->` and `->>` operators with spaces before and after
42+
- Include resource identifiers in non-aggregate queries
43+
- Add to relevant benchmarks (e.g., all controls benchmark, service-specific benchmark)
44+
- Include comprehensive documentation:
45+
- Clear description of what the control checks
46+
- Explanation of why this check is important
47+
- List of variables and their purposes
48+
- Example queries and expected results
49+
- Remediation steps for failing resources
50+
- Any limitations or important notes
4251
```
4352

44-
### Create and Test the Query
53+
### Run Control
4554

46-
Next, develop the SQL query that will power your control:
55+
Next, verify the control is properly configured:
4756

4857
```md
49-
Your goal is to create a working SQL query for your new control.
50-
51-
1. Follow the query structure conventions identified earlier.
58+
Your goal is to verify the control runs without errors.
5259

53-
2. Test the query using one of these methods:
54-
- Powerpipe MCP server's steampipe_query tool (preferred)
55-
- Steampipe CLI query command
60+
1. If using the Powerpipe MCP server (preferred):
61+
- Set the mod location to the current directory
62+
- Use powerpipe_control_run to execute the control
63+
- Verify the control exists and can be queried
5664

57-
3. Verify the query:
58-
- Returns expected results
59-
- Handles edge cases appropriately
60-
- Follows performance best practices
61-
- Uses consistent operators (e.g., '->' and '->>' with spaces)
65+
2. If not using MCP server:
66+
- Run `powerpipe control run <control_name>`
67+
- Check for any syntax or configuration errors
6268

63-
4. Do not proceed until you have a fully working and tested query.
69+
3. Verify:
70+
- Control executes without errors
71+
- Query syntax is valid
72+
- Variables are properly defined
73+
- Documentation examples are correct
6474
```
6575

66-
### Create the Control
76+
### Create Test Resources
6777

68-
With a working query, create the control definition:
78+
Set up resources to validate control behavior:
6979

7080
```md
71-
Your goal is to implement the control using your tested query.
72-
73-
1. Choose the appropriate directory and file based on conventions.
74-
75-
2. Create the control definition:
76-
- Use descriptive names following conventions
77-
- Include clear descriptions and documentation
78-
- Set appropriate severity levels
79-
- Define any necessary variables
80-
- Reference your tested query
81-
82-
3. Handle variables properly:
83-
- Place non-shared variables in the control file
84-
- Only use variables.pp for shared variables
85-
- Set appropriate default values
81+
Your goal is to create test resources that will trigger both passing and failing results.
82+
83+
1. Create test resources:
84+
- Use provider's CLI if available
85+
- Use Terraform if CLI isn't available
86+
- Use API calls via shell script as a last resort
87+
- Create resources that should:
88+
- Pass the control requirements
89+
- Fail the control requirements
90+
- Test edge cases
91+
- Use cost-effective configurations
92+
- Create any dependent resources needed
93+
94+
2. Document the test resources:
95+
- Resource identifiers
96+
- Expected control results
97+
- Steps to recreate
98+
- Cleanup instructions
8699
```
87100

88-
### Add Documentation
101+
### Validate Control Results
89102

90-
Document your new control thoroughly:
103+
Verify the control produces correct results:
91104

92105
```md
93-
Your goal is to create comprehensive documentation for your control.
106+
Your goal is to validate the control produces expected results for test resources.
94107

95-
1. Choose the appropriate documentation location based on conventions.
108+
1. Run the control:
109+
- Use Powerpipe MCP server's powerpipe_control_run tool (preferred)
110+
- Or use Powerpipe CLI: powerpipe control run <control_name>
96111

97-
2. Include:
98-
- Clear description of what the control checks
99-
- Explanation of why this check is important
100-
- List of variables and their purposes
101-
- Example queries and expected results
102-
- Remediation steps for failing resources
103-
- Any limitations or important notes
104-
105-
3. Follow existing documentation format and style.
112+
2. Verify:
113+
- Passing resources are correctly identified
114+
- Failing resources are correctly identified
115+
- Edge cases are handled properly
116+
- Result messages are clear and actionable
117+
- All example queries from documentation execute successfully
118+
- Results match example descriptions
119+
120+
3. Document all test results in raw Markdown format for easy review
106121
```
107122

108-
### Test the Control
123+
### Cleanup Test Resources
109124

110-
Finally, test your complete control implementation:
125+
Finally, clean up your test environment:
111126

112127
```md
113-
Your goal is to verify your control works correctly end-to-end.
128+
Your goal is to remove all test resources to avoid ongoing costs.
114129

115-
1. Test using one of these methods:
116-
- Powerpipe MCP server's powerpipe_control_run tool (preferred)
117-
- First set mod-location to current directory
118-
- Powerpipe CLI command: powerpipe control run <control_name>
130+
1. Delete all resources created for testing:
131+
- Use the same method used for creation
132+
- Remove any dependent resources
133+
- Follow provider's recommended deletion order
119134

120-
2. Verify:
121-
- Control executes without errors
122-
- Results match expectations
123-
- Documentation examples work correctly
124-
- Variables function as intended
125-
126-
3. Make any necessary adjustments and retest until everything works perfectly.
135+
2. Verify cleanup:
136+
- Confirm all resources are successfully deleted
137+
- Run the control again to verify 0 results or expected baseline
127138
```
128139

140+
## Powerpipe HCL
141+
142+
For detailed information about writing Powerpipe HCL configurations, including benchmarks, controls, dashboards, and more, see the [Powerpipe HCL documentation](../develop/hcl).
143+
129144
Remember that AI tools work best when they can see the full context of your mod repository. Always open the entire repository in your IDE and use the Powerpipe MCP server when available for the most efficient development experience.

0 commit comments

Comments
 (0)