-
Notifications
You must be signed in to change notification settings - Fork 510
feat: add Agentless Hello World integration #15729
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
071873c
062af80
48a3802
c8daacf
0d69977
4a8bdf2
8b39cac
ac4b894
0c4599e
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,3 @@ | ||
| dependencies: | ||
| ecs: | ||
| reference: git@v9.1.0 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| rules: | ||
| - path: / | ||
| methods: ["GET"] | ||
| responses: | ||
| - status_code: 418 | ||
| headers: | ||
| Content-Type: | ||
| - "application/json" | ||
| body: |- | ||
| {"this_is": "ignored"} |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,14 @@ | ||
| services: | ||
| epr_mock: | ||
| image: docker.elastic.co/observability/stream:v0.18.0 | ||
| hostname: epr_mock | ||
| ports: | ||
| - 8080 | ||
| volumes: | ||
| - ./config.yml:/config.yml | ||
| environment: | ||
| PORT: "8080" | ||
| command: | ||
| - http-server | ||
| - --addr=:8080 | ||
| - --config=/config.yml |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| - version: "0.1.0" | ||
| changes: | ||
| - description: Initial release. | ||
| type: enhancement | ||
| link: https://github.com/elastic/integrations/pull/15729 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| { | ||
| "events": [ | ||
| { | ||
| "message": "{\"status_code\":200}" | ||
| } | ||
| ] | ||
| } |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| fields: {} |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,23 @@ | ||
| { | ||
| "expected": [ | ||
| { | ||
| "agentless_hello_world": { | ||
| "generic": { | ||
| "status_code": 200 | ||
| } | ||
| }, | ||
| "ecs": { | ||
| "version": "9.1.0" | ||
| }, | ||
| "event": { | ||
| "category": [ | ||
| "web" | ||
| ], | ||
| "kind": "event", | ||
| "type": [ | ||
| "info" | ||
| ] | ||
| } | ||
| } | ||
| ] | ||
| } |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| input: cel | ||
| service: epr_mock | ||
| data_stream: | ||
| vars: | ||
| url: http://{{Hostname}}:{{Port}} | ||
| assert: | ||
| hit_count: 1 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,18 @@ | ||
| config_version: 2 | ||
| interval: 20s | ||
| resource.timeout: 15s | ||
| resource.url: "{{url}}" | ||
| state: | ||
| url: "{{url}}" | ||
| program: | | ||
| request("GET", state.url) | ||
| .do_request() | ||
| .as(resp, { | ||
| "events": [{ | ||
| "message": { | ||
| "status_code": resp.StatusCode | ||
| }.encode_json() | ||
| }] | ||
| }) | ||
| tags: | ||
| - agentless-hello-world |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,88 @@ | ||
| --- | ||
| description: Pipeline for processing Agentless Hello World generic logs. | ||
| processors: | ||
| - set: | ||
| field: ecs.version | ||
| value: '9.1.0' | ||
| - remove: | ||
| field: | ||
| - organization | ||
| - division | ||
| - team | ||
| ignore_missing: true | ||
| if: ctx.organization instanceof String && ctx.division instanceof String && ctx.team instanceof String | ||
| tag: remove_agentless_tags | ||
| description: >- | ||
| Removes the fields added by Agentless as metadata, as they can collide with ECS fields. | ||
| - terminate: | ||
| tag: data_collection_error | ||
| if: ctx.error?.message != null && ctx.message == null && ctx.event?.original == null | ||
| - rename: | ||
| field: message | ||
| target_field: event.original | ||
| ignore_missing: true | ||
| if: ctx.event?.original == null | ||
| - remove: | ||
| field: message | ||
| tag: remove_message | ||
| ignore_missing: true | ||
| description: The `message` field is no longer required if the document has an `event.original` field. | ||
| if: ctx.event?.original != null | ||
| - json: | ||
| field: event.original | ||
| target_field: agentless_hello_world.generic | ||
| on_failure: | ||
| - set: | ||
| field: error.type | ||
| value: "json_parse_error" | ||
| - set: | ||
| field: error.message | ||
| value: "{{{ _ingest.on_failure_message }}}" | ||
| - set: | ||
| field: event.kind | ||
| value: event | ||
| - set: | ||
| field: event.type | ||
| value: [info] | ||
| - set: | ||
| field: event.category | ||
| value: [web] | ||
| - remove: | ||
| field: event.original | ||
| if: ctx.tags == null || !(ctx.tags.contains('preserve_original_event')) | ||
| ignore_failure: true | ||
| ignore_missing: true | ||
| - script: | ||
| lang: painless | ||
| description: This script processor iterates over the whole document to remove fields with null values. | ||
| source: | | ||
| void handleMap(Map map) { | ||
| for (def x : map.values()) { | ||
| if (x instanceof Map) { | ||
| handleMap(x); | ||
| } else if (x instanceof List) { | ||
| handleList(x); | ||
| } | ||
| } | ||
| map.values().removeIf(v -> v == null); | ||
| } | ||
| void handleList(List list) { | ||
| for (def x : list) { | ||
| if (x instanceof Map) { | ||
| handleMap(x); | ||
| } else if (x instanceof List) { | ||
| handleList(x); | ||
| } | ||
| } | ||
| } | ||
| handleMap(ctx); | ||
| on_failure: | ||
| - set: | ||
| field: event.kind | ||
| value: pipeline_error | ||
| - append: | ||
| field: error.message | ||
| value: >- | ||
| Processor '{{{ _ingest.on_failure_processor_type }}}' | ||
| {{#_ingest.on_failure_processor_tag}}with tag '{{{ _ingest.on_failure_processor_tag }}}' | ||
| {{/_ingest.on_failure_processor_tag}}failed with message '{{{ _ingest.on_failure_message }}}' | ||
| Original file line number | Diff line number | Diff line change | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| @@ -0,0 +1,23 @@ | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| - name: input.type | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| type: keyword | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| description: Input type | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| - name: data_stream.type | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| type: constant_keyword | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| description: Data stream type. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| - name: data_stream.dataset | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| type: constant_keyword | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| description: Data stream dataset. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| - name: data_stream.namespace | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| type: constant_keyword | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| description: Data stream namespace. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| - name: event.module | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| type: constant_keyword | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| description: Event module | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| value: agentless_hello_world | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| - name: event.dataset | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| type: constant_keyword | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| description: Event dataset | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| value: agentless_hello_world.generic | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| - name: '@timestamp' | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| type: date | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| description: Event timestamp. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Comment on lines
+4
to
+23
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Since these fields exist in ECS, I recommend to depend the ECS definition for consistency. (More background in https://github.com/elastic/integrations/wiki/Fleet-Package-Code-Review-Comments#defining-an-ecs-field-without-using-an-external-definition)
Suggested change
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| - name: agentless_hello_world.generic | ||
| type: group | ||
| fields: | ||
| - name: status_code | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. You could use ECS http.response.status_code (https://www.elastic.co/docs/reference/ecs/ecs-http#field-http-response-status-code) for this field instead. |
||
| type: long | ||
| description: HTTP Status Code | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,15 @@ | ||
| title: Generic logs | ||
| type: logs | ||
| streams: | ||
| - input: cel | ||
| title: Generic logs | ||
| description: Collect generic logs from EPR endpoint. | ||
| template_path: cel.yml.hbs | ||
| vars: | ||
| - name: url | ||
| type: text | ||
| title: EPR URL | ||
| description: URL of the EPR endpoint (internal use only, for testing) | ||
| default: https://epr.elastic.co | ||
| required: false | ||
| show_user: false |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,69 @@ | ||
| { | ||
| "@timestamp": "2025-10-23T11:25:00.349Z", | ||
| "agent": { | ||
| "ephemeral_id": "2f0402ea-00e1-47fa-944c-1e34d91fdc2f", | ||
| "id": "0909c464-2093-4f85-8bf7-b11593587146", | ||
| "name": "elastic-agent-93305", | ||
| "type": "filebeat", | ||
| "version": "9.1.3" | ||
| }, | ||
| "agentless_hello_world": { | ||
| "generic": { | ||
| "status_code": 418 | ||
| } | ||
| }, | ||
| "data_stream": { | ||
| "dataset": "agentless_hello_world.generic", | ||
| "namespace": "88559", | ||
| "type": "logs" | ||
| }, | ||
| "ecs": { | ||
| "version": "9.1.0" | ||
| }, | ||
| "elastic_agent": { | ||
| "id": "0909c464-2093-4f85-8bf7-b11593587146", | ||
| "snapshot": false, | ||
| "version": "9.1.3" | ||
| }, | ||
| "event": { | ||
| "agent_id_status": "verified", | ||
| "category": [ | ||
| "web" | ||
| ], | ||
| "dataset": "agentless_hello_world.generic", | ||
| "ingested": "2025-10-23T11:25:03Z", | ||
| "kind": "event", | ||
| "module": "agentless_hello_world", | ||
| "type": [ | ||
| "info" | ||
| ] | ||
| }, | ||
| "host": { | ||
| "architecture": "aarch64", | ||
| "containerized": false, | ||
| "hostname": "elastic-agent-93305", | ||
| "ip": [ | ||
| "172.30.0.2", | ||
| "172.18.0.4" | ||
| ], | ||
| "mac": [ | ||
| "36-F7-E4-8A-31-61", | ||
| "B2-C0-07-A9-21-9B" | ||
| ], | ||
| "name": "elastic-agent-93305", | ||
| "os": { | ||
| "family": "", | ||
| "kernel": "6.10.14-linuxkit", | ||
| "name": "Wolfi", | ||
| "platform": "wolfi", | ||
| "type": "linux", | ||
| "version": "20230201" | ||
| } | ||
| }, | ||
| "input": { | ||
| "type": "cel" | ||
| }, | ||
| "tags": [ | ||
| "agentless-hello-world" | ||
| ] | ||
| } |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,53 @@ | ||
| # Agentless Hello World | ||
|
|
||
| This is a sample integration designed to exercise the Agentless infrastructure. It periodically fetches data from `https://epr.elastic.co` every minute to demonstrate basic agentless functionality. | ||
|
|
||
| ## Overview | ||
|
|
||
| The Agentless Hello World integration is a minimal example that: | ||
| - Fetches data from the Elastic Package Registry (EPR) endpoint | ||
| - Runs every 1 minute | ||
| - Requires no user configuration | ||
|
|
||
| ## Configuration | ||
|
|
||
| This integration requires no configuration from the user. All settings are pre-configured: | ||
| - **Endpoint**: `https://epr.elastic.co` | ||
| - **Interval**: 1 minute | ||
| - **Deployment mode**: Agentless by default | ||
|
|
||
| ## Data Collection | ||
|
|
||
| The integration makes HTTP GET requests to `https://epr.elastic.co` and stores: | ||
| - **status_code**: HTTP Status Code for the response. | ||
|
|
||
| ## Requirements | ||
|
|
||
| ### Agentless-enabled integration | ||
|
|
||
| Agentless integrations allow you to collect data without having to manage Elastic Agent in your cloud. They make manual agent deployment unnecessary, so you can focus on your data instead of the agent that collects it. For more information, refer to [Agentless integrations](https://www.elastic.co/guide/en/serverless/current/security-agentless-integrations.html) and the [Agentless integrations FAQ](https://www.elastic.co/guide/en/serverless/current/agentless-integration-troubleshooting.html). | ||
|
|
||
| Agentless deployments are only supported in Elastic Serverless and Elastic Cloud environments. This functionality is in beta and is subject to change. Beta features are not subject to the support SLA of official GA features. | ||
|
|
||
| ## Logs | ||
|
|
||
| ### Generic | ||
|
|
||
| The generic data stream collects responses from the EPR endpoint. | ||
|
|
||
| **ECS Field Reference** | ||
|
|
||
| Please refer to the following document for detailed information on ECS fields: | ||
| - [Elastic Common Schema](https://www.elastic.co/guide/en/ecs/current/ecs-field-reference.html) | ||
|
|
||
| **Exported fields** | ||
|
|
||
| | Field | Description | Type | | ||
| |---|---|---| | ||
| | @timestamp | Event timestamp. | date | | ||
| | agentless_hello_world.generic.status_code | The HTTP Status Code of the response. | long | | ||
| | data_stream.dataset | Data stream dataset. | constant_keyword | | ||
| | data_stream.namespace | Data stream namespace. | constant_keyword | | ||
| | data_stream.type | Data stream type. | constant_keyword | | ||
| | event.dataset | Event dataset | constant_keyword | | ||
| | event.module | Event module | constant_keyword | |
Uh oh!
There was an error while loading. Please reload this page.