Skip to content

Commit 820286a

Browse files
authored
Merge pull request #2642 from port-labs/OCEAN-27-MULTIPLE-WORKERS-DOC-GITHUB-OCEAN
feat: add event_workers_count config for github ocean
2 parents a243baf + 9644387 commit 820286a

File tree

1 file changed

+35
-2
lines changed
  • docs/build-your-software-catalog/sync-data-to-catalog/git/github-ocean/installation

1 file changed

+35
-2
lines changed

docs/build-your-software-catalog/sync-data-to-catalog/git/github-ocean/installation/installation.mdx

Lines changed: 35 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -40,9 +40,9 @@ To create a personal access token see Github's [managing your personal access to
4040

4141
<details>
4242
<summary>Required permissions</summary>
43-
43+
4444
These are the necessary permissions if you're creating a fine-grained PAT.
45-
45+
4646
**Repository permissions:**
4747
- **Content**: Readonly
4848
- **Metadata**: Readonly
@@ -70,4 +70,37 @@ To set it up, see the [GitHub App installation guide](./github-app.mdx).
7070
</TabItem>
7171
</Tabs>
7272

73+
### Configure parallel processing
74+
75+
Configure multiple workers to handle GitHub webhook events in parallel groups, preventing race conditions while maintaining event ordering for related resources.
76+
77+
Starting with Ocean 0.27, a new parameter was added to the configuration, which adjusts the number of async workers spawned for webhook events.
78+
79+
Set `event_workers_count: 4` in your config (default: 1) to enable parallel group processing.
80+
81+
When `event_workers_count > 1`, related GitHub events get grouped together:
82+
- All PR events (opened, review, status) for PR #123 → same processing group.
83+
- All issue events for issue #456 → same processing group.
84+
- Push events grouped by commit SHA.
85+
86+
<details>
87+
<summary>Event grouping behavior</summary>
88+
89+
**Supported event types:**
90+
- **Pull Requests**: Grouped by PR number (`pull_request`, `pull_request_review`, `pull_request_review_comment`)
91+
- **Issues**: Grouped by issue number (`issues`, `issue_comment`)
92+
- **Pushes**: Grouped by commit SHA (`push`)
93+
- **Releases**: Grouped by release ID (`release`)
94+
- **Workflow Runs**: Grouped by run ID (`workflow_run`)
95+
- **Status Checks**: Grouped by commit SHA (`status`)
96+
97+
**Fallback**: If the event type isn't recognized, the payload will be scanned for the first `number`, `id`, or `sha` field.
98+
99+
:::note Processing guarantees
100+
Events within the same group are processed sequentially to maintain order and prevent conflicts.
101+
Different groups can be processed in parallel across workers.
102+
With `event_workers_count: 1`, all events are processed sequentially using a simple queue.
103+
:::
104+
</details>
105+
73106
<InstallMethods setupName="classic" />

0 commit comments

Comments
 (0)