Skip to content

Update OpenAPI schema for /crawl/active endpoint created_at field #47

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversation

devin-ai-integration[bot]
Copy link
Contributor

@devin-ai-integration devin-ai-integration bot commented Jun 26, 2025

Add created_at field to /crawl/active endpoint response

Summary

Adds a new created_at field to the /crawl/active API endpoint response, showing when each crawl was created. The field returns an ISO 8601 timestamp string converted from the existing Unix timestamp stored in Redis.

Key Changes:

  • Updated OngoingCrawlsResponse TypeScript type to include created_at: string
  • Modified ongoingCrawlsController to return created_at field by converting Unix timestamp to ISO string
  • Updated OpenAPI schema in firecrawl-docs repository to document the new field
  • Added comprehensive tests validating field presence, format, and recency

Review & Testing Checklist for Human

  • Verify endpoint actually returns created_at field - Test the live /crawl/active endpoint to confirm the field appears in responses
  • Check timestamp format and accuracy - Ensure timestamps are valid ISO 8601 strings and match actual crawl creation times
  • Test fallback behavior - Verify what happens when createdAt is undefined/null on stored crawl objects (currently falls back to Date.now())
  • Validate no regressions - Test existing crawl functionality to ensure no breaking changes
  • Confirm OpenAPI schema alignment - Verify the documented schema in firecrawl-docs matches the actual implementation

Recommended test plan:

  1. Start a new crawl via API
  2. Immediately call /crawl/active to see the crawl with created_at field
  3. Verify timestamp is recent and properly formatted
  4. Test with multiple concurrent crawls to ensure consistent behavior

Diagram

graph TD
    A["/crawl/active endpoint"] --> B["ongoingCrawlsController"]
    B --> C["getCrawls from Redis"]
    C --> D["StoredCrawl objects"]
    D --> E["createdAt: number"]
    
    B --> F["Response mapping"]
    F --> G["new Date(x.createdAt || Date.now()).toISOString()"]
    G --> H["created_at: string in response"]
    
    I["OngoingCrawlsResponse type"] --> F
    J["OpenAPI schema"] --> K["API documentation"]
    
    L["crawl.test.ts"] --> M["Test coverage"]
    
    B:::major-edit
    F:::major-edit
    I:::major-edit
    J:::major-edit
    L:::major-edit
    
    A:::context
    C:::context
    D:::context
    E:::context
    H:::context
    K:::context
    M:::context
    
    subgraph Legend
        L1[Major Edit]:::major-edit
        L2[Minor Edit]:::minor-edit  
        L3[Context/No Edit]:::context
    end
    
    classDef major-edit fill:#90EE90
    classDef minor-edit fill:#87CEEB
    classDef context fill:#FFFFFF
Loading

Notes

  • Cross-repo dependency: This PR requires coordinated deployment with the firecrawl-docs PR Update OpenAPI schema for /crawl/active endpoint created_at field #47 that updates the OpenAPI schema
  • Fallback behavior: When createdAt is undefined, it falls back to Date.now() - this may not be the desired behavior for all use cases
  • Manual testing confirmed: Tested locally and confirmed the endpoint returns the new field with proper ISO timestamp format
  • Test environment limitations: Some tests failed due to Supabase configuration issues in the test environment, but core functionality tests passed

- Add created_at field to crawl object schema
- Update required fields array to include created_at
- Corresponds to firecrawl API implementation changes

Co-Authored-By: rafael@sideguide.dev <rafael@sideguide.dev>
Copy link
Contributor Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment and CI monitoring

Copy link
Contributor Author

Closing due to inactivity for more than 30 days. Configure here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

0 participants