Skip to content

Commit 645be58

Browse files
committed
Add documentation for using AI to create controls in Powerpipe mods
1 parent e4130da commit 645be58

File tree

1 file changed

+148
-0
lines changed

1 file changed

+148
-0
lines changed

docs/develop/using-ai.md

Lines changed: 148 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,148 @@
1+
---
2+
title: Using AI to Create Controls
3+
sidebar_label: Using AI for Controls
4+
---
5+
6+
# Using AI to Create Controls
7+
8+
Creating new controls for Powerpipe mods with AI tools and IDEs works remarkably well. At Turbot, we develop controls frequently and use AI for almost every new control we create. We've found that AI typically produces excellent results when working with existing mod repositories, as it can learn from established patterns and conventions.
9+
10+
The key to success is working within existing mod repositories and opening the entire repository as a folder or project in your IDE. This gives AI tools access to existing control implementations, documentation examples, code patterns, and naming conventions to generate consistent, high-quality results.
11+
12+
If you're looking to use AI to run Powerpipe controls rather than develop new ones, you can use the [Powerpipe MCP server](../query/mcp), which provides powerful tools for AI agents to inspect and run controls, benchmarks, and queries.
13+
14+
## Getting Started
15+
16+
While AI often works well with simple requests like "Create a control for [requirement]", here are some structured prompts and workflows we use at Turbot that you may find helpful as starting points.
17+
18+
### Prerequisites
19+
20+
1. Open the mod repository in your IDE (Cursor, VS Code, Windsurf, etc.) to give AI tools access to all existing code and documentation.
21+
2. Ensure you have Powerpipe installed with all necessary plugins configured.
22+
3. Set up access to the cloud providers or systems you'll be evaluating.
23+
4. Configure the [Powerpipe MCP server](https://github.com/turbot/steampipe-mcp) which allows the agent to inspect and run controls.
24+
25+
### Step 1: Understand Conventions
26+
27+
First, review existing controls and conventions to ensure consistency:
28+
29+
```md
30+
Your goal is to understand the conventions for creating a new Powerpipe control.
31+
32+
1. Review the sample repository at https://github.com/turbot/steampipe-mod-aws-thrifty for reference patterns.
33+
34+
2. Analyze existing controls in the current mod to understand:
35+
- Control naming conventions
36+
- File and directory organization
37+
- Query structure and style
38+
- Variable naming and placement
39+
- Documentation format and requirements
40+
41+
3. Document the key conventions you've identified to ensure your new control will be consistent.
42+
```
43+
44+
### Step 2: Create and Test the Query
45+
46+
Next, develop the SQL query that will power your control:
47+
48+
```md
49+
Your goal is to create a working SQL query for your new control.
50+
51+
1. Follow the query structure conventions identified in Step 1.
52+
53+
2. Test the query using one of these methods:
54+
- Powerpipe MCP server's steampipe_query tool (preferred)
55+
- Steampipe CLI query command
56+
57+
3. Verify the query:
58+
- Returns expected results
59+
- Handles edge cases appropriately
60+
- Follows performance best practices
61+
- Uses consistent operators (e.g., '->' and '->>' with spaces)
62+
63+
4. Do not proceed until you have a fully working and tested query.
64+
```
65+
66+
### Step 3: Create the Control
67+
68+
With a working query, create the control definition:
69+
70+
```md
71+
Your goal is to implement the control using your tested query.
72+
73+
1. Choose the appropriate directory and file based on conventions.
74+
75+
2. Create the control definition:
76+
- Use descriptive names following conventions
77+
- Include clear descriptions and documentation
78+
- Set appropriate severity levels
79+
- Define any necessary variables
80+
- Reference your tested query
81+
82+
3. Handle variables properly:
83+
- Place non-shared variables in the control file
84+
- Only use variables.pp for shared variables
85+
- Set appropriate default values
86+
```
87+
88+
### Step 4: Add Documentation
89+
90+
Document your new control thoroughly:
91+
92+
```md
93+
Your goal is to create comprehensive documentation for your control.
94+
95+
1. Choose the appropriate documentation location based on conventions.
96+
97+
2. Include:
98+
- Clear description of what the control checks
99+
- Explanation of why this check is important
100+
- List of variables and their purposes
101+
- Example queries and expected results
102+
- Remediation steps for failing resources
103+
- Any limitations or important notes
104+
105+
3. Follow existing documentation format and style.
106+
```
107+
108+
### Step 5: Test the Control
109+
110+
Finally, test your complete control implementation:
111+
112+
```md
113+
Your goal is to verify your control works correctly end-to-end.
114+
115+
1. Test using one of these methods:
116+
- Powerpipe MCP server's powerpipe_control_run tool (preferred)
117+
- First set mod-location to current directory
118+
- Powerpipe CLI command: powerpipe control run <control_name>
119+
120+
2. Verify:
121+
- Control executes without errors
122+
- Results match expectations
123+
- Documentation examples work correctly
124+
- Variables function as intended
125+
126+
3. Make any necessary adjustments and retest until everything works perfectly.
127+
```
128+
129+
## Best Practices
130+
131+
1. **Start Small**: Begin with simple controls and gradually add complexity as needed.
132+
2. **Test Thoroughly**: Always test your query independently before implementing the control.
133+
3. **Follow Patterns**: Stick to established conventions from existing controls.
134+
4. **Document Clearly**: Write documentation that helps users understand both what the control does and why it's important.
135+
5. **Consider Performance**: Optimize queries for large-scale environments.
136+
6. **Use Variables**: Make controls flexible with variables, but keep defaults sensible.
137+
7. **Think Modular**: Design controls that can be reused across different benchmarks.
138+
139+
## Common Pitfalls to Avoid
140+
141+
1. **Overly Complex Queries**: Keep queries focused and simple when possible.
142+
2. **Missing Edge Cases**: Test with various resource configurations.
143+
3. **Hardcoded Values**: Use variables for configurable thresholds.
144+
4. **Incomplete Documentation**: Always include examples and remediation steps.
145+
5. **Inconsistent Naming**: Follow established naming patterns.
146+
6. **Poor Error Handling**: Consider and handle potential error conditions.
147+
148+
Remember that AI tools work best when they can see the full context of your mod repository. Always open the entire repository in your IDE and use the Powerpipe MCP server when available for the most efficient development experience.

0 commit comments

Comments
 (0)