Merge branch 'feature/permission-boundaries' into 'develop'

Feature/permission boundaries

See merge request genaiic-reusable-assets/engagement-artifacts/genaiic-idp-accelerator!235

Author: rstrahan

CHANGELOG.md

@@ -5,6 +5,12 @@ SPDX-License-Identifier: MIT-0
 
 ## [Unreleased]
 
+### Added
+- **Optional Permissions Boundary Support for Enterprise Deployments**
+  - Added `PermissionsBoundaryArn` parameter to all CloudFormation templates for organizations with Service Control Policies (SCPs) requiring permissions boundaries
+  - Comprehensive support for both explicit IAM roles and implicit roles created by AWS SAM functions and statemachines`
+  - Conditional implementation ensures backward compatibility - when no permissions boundary is provided, roles deploy normally
+
 ## [0.3.8]
 
 ### Added

docs/aws-services-and-roles.md

@@ -51,6 +51,20 @@ This document outlines the AWS services used by the GenAI Intelligent Document P
 
 ## IAM Role Requirements
 
+### Enterprise Deployment Considerations
+
+For organizations with Service Control Policies (SCPs) that mandate permissions boundaries on all IAM roles, the solution provides comprehensive support through the `PermissionsBoundaryArn` parameter. This optional parameter can be specified during deployment to attach a permissions boundary to all IAM roles (both explicit roles and implicit roles created by AWS SAM functions).
+
+**Usage:**
+```bash
+aws cloudformation deploy \
+  --template-file template.yaml \
+  --parameter-overrides PermissionsBoundaryArn=arn:aws:iam::123456789012:policy/MyPermissionsBoundary \
+  --capabilities CAPABILITY_IAM
+```
+
+When no permissions boundary is specified, roles deploy normally, ensuring backward compatibility.
+
 ### Deployment Roles
 
 Deploying this solution requires an IAM role/user with the following permissions:

docs/well-architected.md

@@ -30,6 +30,7 @@ The GenAI Intelligent Document Processing (GenAIIDP) Accelerator demonstrates st
 ### Strengths
 
 - **Defense in Depth**: Multiple security layers including IAM roles with least privilege, encryption at rest, and secure API access.
+- **Enterprise IAM Governance**: Comprehensive support for IAM permissions boundaries to comply with organizational Service Control Policies (SCPs) that mandate permissions boundaries on all IAM roles.
 - **Content Safety**: Integration with Amazon Bedrock Guardrails to enforce content policies, block sensitive information, and prevent model misuse.
 - **Authentication**: Cognito user pools with configurable password policies and MFA support.
 - **Authorization**: Fine-grained access controls for different components and resources.
@@ -146,4 +147,4 @@ The GenAI Intelligent Document Processing Accelerator demonstrates strong alignm
 
 Key strengths include the serverless architecture, which provides automatic scaling and resilience, and the comprehensive monitoring capabilities that enable operational visibility. The solution's modular design allows for customization and extension to meet specific business requirements.
 
-Areas for potential enhancement include more granular cost controls, multi-region resilience strategies, and sustainability optimizations. By addressing these recommendations, the solution can further improve its alignment with Well-Architected best practices.
+Areas for potential enhancement include more granular cost controls, multi-region resilience strategies, and sustainability optimizations. By addressing these recommendations, the solution can further improve its alignment with Well-Architected best practices.

memory-bank/activeContext.md

@@ -2,111 +2,85 @@
 
 ## Current Task Focus
 
-**User Question**: Understanding OCR processing architecture for large PDFs (500+ pages) in the IDP accelerator, specifically:
-1. Is OCR processing sequential or distributed by page?
-2. How does Bedrock-only OCR deployment differ?
-3. What parts of the system run sequentially vs distributed?
-4. Handling massive PDFs with hundreds of forms without clear page boundaries
+**Customer Question**: "We are encountering difficulties deploying your IDP stack outside of a sandbox environment due to an organization-wide Service Control Policy (SCP). This policy mandates the attachment of a Permissions Boundary to any new role. Could you please inform us if it is possible to update the CloudFormation template to include a parameterized Permissions Boundary? Without this update, our ability to transition the code to production will be significantly impeded."
 
-## Key Findings
+**Task Status**: Implementation phase - Need to add Permissions Boundary parameter support to CloudFormation templates
 
-### OCR Processing Models
+## Problem Analysis
 
-The IDP accelerator uses **different processing models depending on the pattern**:
+### Current Situation
+- IDP stack creates numerous IAM roles across main template and pattern templates
+- Organization has SCP requiring Permissions Boundary on all new IAM roles
+- Current templates don't support Permissions Boundary configuration
+- Blocking production deployment
 
-#### Pattern 1 (BDA): Sequential Internal Processing
-- **OCR Approach**: Bedrock Data Automation handles everything internally
-- **Processing**: Entire document processed as single unit by BDA service
-- **Concurrency**: Not user-controllable, managed by BDA
-- **Large Documents**: Subject to BDA service limits and timeouts
+### Affected Templates
+- **Main Template**: `template.yaml` - ~15 IAM roles
+- **Pattern 1**: `patterns/pattern-1/template.yaml` - ~8 IAM roles  
+- **Pattern 2**: `patterns/pattern-2/template.yaml` - ~6 roles
+- **Pattern 3**: `patterns/pattern-3/template.yaml` - ~5 roles
+- **Options**: `options/bda-lending-project/template.yaml`, `options/bedrockkb/template.yaml`
 
-#### Pattern 2/3 (Textract + Bedrock): Distributed Page Processing
-- **OCR Approach**: AWS Textract with concurrent page processing
-- **Processing**: **Pages processed in parallel** using ThreadPoolExecutor
-- **Concurrency**: Configurable (default: 20 concurrent workers)
-- **Large Documents**: Optimal for 500+ page documents
+## Solution Design
 
-### Sequential vs Distributed Components
+### Approach: Parameterized Permissions Boundary
+1. **Add optional parameter** to main template for Permissions Boundary ARN
+2. **Conditionally apply boundary** to all IAM roles when provided
+3. **Maintain backward compatibility** for deployments without boundaries
+4. **Cascade parameter** to all nested pattern stacks
 
-#### Sequential Processing:
-1. **Step Functions Workflow**: OCR → Classification → Extraction → Assessment → Summarization
-2. **Classification**: Analyzes all pages to create document boundaries
-3. **BDA Internal Processing**: Everything handled as single unit
+### Implementation Plan
 
-#### Distributed Processing:
-1. **OCR Pages (Pattern 2/3)**: Up to 20 pages processed simultaneously
-2. **Extraction Sections**: Up to 10 document sections processed in parallel
-3. **Independent API Calls**: Each page makes separate Textract calls
+#### Step 1: Main Template Updates (`template.yaml`)
+- Add `PermissionsBoundaryArn` parameter
+- Add `HasPermissionsBoundary` condition
+- Update all IAM role resources with conditional boundary
+- Pass parameter to nested stacks
+- Update CloudFormation interface metadata
 
-## Customer Scenario Analysis
+#### Step 2: Pattern Template Updates
+- Add parameter to each pattern template
+- Update all IAM roles in patterns
+- Maintain consistency across all patterns
 
-### 500+ Page PDF with Multiple Forms
+#### Step 3: Options Template Updates
+- Update BDA lending project template
+- Update Bedrock KB template
 
-**Challenge**: Single PDF containing hundreds of forms without clear page boundaries
+### Key Implementation Details
 
-**Recommended Approach**: Pattern 2 or 3 for optimal performance
-
-**Why Pattern 2/3 is Better**:
-- **Page-Level Parallelism**: 500 pages processed 20 at a time
-- **Memory Efficiency**: Individual pages loaded, not entire document
-- **Fault Tolerance**: Page failures don't stop entire processing
-- **Granular Control**: Can optimize per-page processing
-
-**Classification Strategy**:
-- Use "holistic" classification method to analyze entire document
-- Creates logical sections grouping related pages
-- Handles form boundaries that don't align with page boundaries
-
-## Technical Implementation Details
+**Parameter Definition:**
+```yaml
+PermissionsBoundaryArn:
+  Type: String
+  Default: ""
+  Description: (Optional) ARN of IAM Permissions Boundary policy
+  AllowedPattern: "^(|arn:aws:iam::[0-9]{12}:policy/.+)$"
+```
 
-### OCR Service Configuration for Large Documents
+**Condition:**
+```yaml
+HasPermissionsBoundary: !Not [!Equals [!Ref PermissionsBoundaryArn, ""]]
+```
 
+**Role Update Pattern:**
 ```yaml
-ocr:
-  backend: "textract"
-  max_workers: 20  # Increase for more parallelism
-  image:
-    dpi: 150      # Balance quality vs processing time
-    target_width: 1024
-    target_height: 1024
-  features:
-    - name: "LAYOUT"
-    - name: "TABLES" 
-    - name: "FORMS"
+SomeRole:
+  Type: AWS::IAM::Role
+  Properties:
+    # existing properties...
+    PermissionsBoundary: !If [HasPermissionsBoundary, !Ref PermissionsBoundaryArn, !Ref AWS::NoValue]
 ```
 
-### Processing Flow for Large PDFs
-
-1. **Document Load**: PyMuPDF loads PDF structure
-2. **Page Distribution**: ThreadPoolExecutor creates 20 concurrent workers
-3. **Parallel OCR**: Each page processed independently via Textract
-4. **Result Assembly**: Pages sorted and combined into document structure
-5. **Classification**: Holistic analysis creates logical document sections
-6. **Parallel Extraction**: Sections processed concurrently (MaxConcurrency: 10)
-
-## Performance Implications
-
-### For 500-Page Document:
-- **Pattern 1 (BDA)**: Single job, BDA-managed processing
-- **Pattern 2/3**: ~25 batches of 20 pages each, highly parallelized
-
-### Bottlenecks to Consider:
-1. **Textract Rate Limits**: May need to adjust max_workers
-2. **Memory Usage**: 20 concurrent pages require significant memory
-3. **S3 Operations**: Parallel uploads/downloads for page results
-4. **Lambda Timeouts**: Ensure sufficient timeout for large documents
-
-## Next Steps and Considerations
-
-### For Customer Implementation:
-1. **Choose Pattern 2 or 3** for large document processing
-2. **Configure max_workers** based on Textract limits and memory
-3. **Use holistic classification** to handle form boundaries
-4. **Monitor memory usage** during processing
-5. **Consider document splitting** if single PDF approach is problematic
-
-### Optimization Opportunities:
-- **Adaptive Concurrency**: Adjust workers based on document size
-- **Progressive Processing**: Start classification while OCR continues
-- **Caching Strategy**: Cache page images for reprocessing
-- **Error Recovery**: Implement page-level retry with exponential backoff
+## Benefits
+- **SCP Compliance**: Satisfies organizational requirements
+- **Backward Compatible**: Existing deployments unaffected
+- **Flexible**: Works with any Permissions Boundary policy
+- **Comprehensive**: Covers all IAM roles across all components
+
+## Next Steps
+1. Implement main template changes
+2. Update all pattern templates
+3. Update options templates
+4. Test deployment scenarios
+5. Document usage examples

options/bda-lending-project/template.yaml

@@ -27,6 +27,20 @@ Parameters:
       - CRITICAL
     Description: Default logging level for Lambda functions
 
+  PermissionsBoundaryArn:
+    Type: String
+    Default: ""
+    Description: >-
+      (Optional) ARN of an existing IAM Permissions Boundary policy to attach to all IAM roles.
+      Required by some organizations with Service Control Policies (SCPs).
+      Format: arn:aws:iam::account-id:policy/policy-name
+      Leave blank if no Permissions Boundary is required.
+    AllowedPattern: "^(|arn:aws:iam::[0-9]{12}:policy/.+)$"
+    ConstraintDescription: Must be empty or a valid IAM policy ARN
+
+Conditions:
+  HasPermissionsBoundary: !Not [!Equals [!Ref PermissionsBoundaryArn, ""]]
+
 Resources:
 
   # IAM role for Lambda function
@@ -47,6 +61,7 @@ Resources:
             Action: sts:AssumeRole
       ManagedPolicyArns:
         - arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole
+      PermissionsBoundary: !If [HasPermissionsBoundary, !Ref PermissionsBoundaryArn, !Ref AWS::NoValue]
       Policies:
         - PolicyName: BedrockDataAutomationAccess
           PolicyDocument:
@@ -143,4 +158,4 @@ Outputs:
 
   BlueprintArns:
     Description: ARNs of the blueprints added to the project
-    Value: !Join [", ", !GetAtt BDAProject.blueprintArns]
+    Value: !Join [", ", !GetAtt BDAProject.blueprintArns]

options/bedrockkb/template.yaml

@@ -123,6 +123,17 @@ Parameters:
     Type: String
     Default: AMAZON_BEDROCK_TEXT_CHUNK
 
+  PermissionsBoundaryArn:
+    Type: String
+    Default: ""
+    Description: >-
+      (Optional) ARN of an existing IAM Permissions Boundary policy to attach to all IAM roles.
+      Required by some organizations with Service Control Policies (SCPs).
+      Format: arn:aws:iam::account-id:policy/policy-name
+      Leave blank if no Permissions Boundary is required.
+    AllowedPattern: "^(|arn:aws:iam::[0-9]{12}:policy/.+)$"
+    ConstraintDescription: Must be empty or a valid IAM policy ARN
+
 Metadata:
   AWS::CloudFormation::Interface:
     ParameterGroups:
@@ -228,6 +239,7 @@ Conditions:
     Fn::Or:
       - Condition: IsChunkingStrategyFixed
       - Condition: IsChunkingStrategyDefault
+  HasPermissionsBoundary: !Not [!Equals [!Ref PermissionsBoundaryArn, ""]]
 
 Resources:
   # Custom resource to transform input to lowercase.
@@ -245,6 +257,7 @@ Resources:
     # checkov:skip=CKV_AWS_115: "Function does not require reserved concurrency as it scales based on demand"
     # checkov:skip=CKV_AWS_173: "Environment variables do not contain sensitive data - only configuration values like feature flags and non-sensitive settings"
     Properties:
+      PermissionsBoundary: !If [HasPermissionsBoundary, !Ref PermissionsBoundaryArn, !Ref AWS::NoValue]
       Handler: index.handler
       Runtime: python3.12
       Timeout: 30
@@ -290,6 +303,7 @@ Resources:
     # checkov:skip=CKV_AWS_115: "Function does not require reserved concurrency as it scales based on demand"
     # checkov:skip=CKV_AWS_173: "Environment variables do not contain sensitive data - only configuration values like feature flags and non-sensitive settings"
     Properties:
+      PermissionsBoundary: !If [HasPermissionsBoundary, !Ref PermissionsBoundaryArn, !Ref AWS::NoValue]
       Handler: index.handler
       Runtime: python3.12
       Timeout: 30
@@ -388,6 +402,7 @@ Resources:
             - lambda.amazonaws.com
           Action:
           - sts:AssumeRole
+      PermissionsBoundary: !If [HasPermissionsBoundary, !Ref PermissionsBoundaryArn, !Ref AWS::NoValue]
       Policies:
       - PolicyName: OSSLambdaRoleDefaultPolicy # Reference: https://docs.aws.amazon.com/service-authorization/latest/reference/list_awsx-ray.html
         PolicyDocument:
@@ -465,6 +480,7 @@ Resources:
     # checkov:skip=CKV_AWS_115: "Function does not require reserved concurrency as it scales based on demand"
     # checkov:skip=CKV_AWS_173: "Environment variables do not contain sensitive data - only configuration values like feature flags and non-sensitive settings"
     Properties:
+      PermissionsBoundary: !If [HasPermissionsBoundary, !Ref PermissionsBoundaryArn, !Ref AWS::NoValue]
       Handler: oss_handler.lambda_handler
       MemorySize: 1024
       Role: !GetAtt OpenSearchLambdaExecutionRole.Arn
@@ -515,6 +531,7 @@ Resources:
                 aws:SourceAccount: !Sub ${AWS::AccountId}
               ArnLike:
                 aws:SourceArn: !Sub arn:aws:bedrock:${AWS::Region}:${AWS::AccountId}:knowledge-base/*
+      PermissionsBoundary: !If [HasPermissionsBoundary, !Ref PermissionsBoundaryArn, !Ref AWS::NoValue]
       Policies:
         - PolicyName: bedrock-invoke-model
           PolicyDocument:
@@ -711,6 +728,7 @@ Resources:
             Action: sts:AssumeRole
       ManagedPolicyArns:
         - arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole
+      PermissionsBoundary: !If [HasPermissionsBoundary, !Ref PermissionsBoundaryArn, !Ref AWS::NoValue]
       Policies:
         - PolicyDocument:
             Version: 2012-10-17
@@ -812,6 +830,7 @@ Resources:
             Principal:
               Service: scheduler.amazonaws.com
             Action: sts:AssumeRole
+      PermissionsBoundary: !If [HasPermissionsBoundary, !Ref PermissionsBoundaryArn, !Ref AWS::NoValue]
       Policies:
         - PolicyName: BedrockAgentStartIngestionPolicy
           PolicyDocument: