Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file added assets/workflows/observability/tilebox-jaeger.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
103 changes: 100 additions & 3 deletions workflows/observability/open-telemetry.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -34,10 +34,107 @@ Tilebox Workflows currently supports OpenTelemetry for tracing and logging, with

## Integrations

Tilebox exports telemetry data using the [OpenTelemetry Protocol](https://opentelemetry.io/docs/specs/otlp/).
Tilebox exports telemetry data using the [OpenTelemetry Protocol](https://opentelemetry.io/docs/specs/otlp/). This allows you to send telemetry data to any OpenTelemetry-compatible backend, such as Axiom or Jaeger.

### Axiom

<img src="/assets/workflows/observability/tilebox-axiom.png" alt="Tilebox is pre-integrated with Axiom" />

Axiom, a cloud-based observability and telemetry platform, supports this protocol. Tilebox Workflows has built-in support for Axiom, and the examples and screenshots in this section come from this integration.
Tilebox Workflows has built-in support for Axiom, a cloud-based observability and telemetry platform. The examples and screenshots in this section come from this integration.

To get started, [sign up for a free Axiom account](https://axiom.co/), create an axiom dataset for traces and logs, and generate an API key with ingest permissions for those datasets.

You can then configure Tilebox to export traces and logs to Axiom using [configure_otel_tracing_axiom](/workflows/observability/tracing#axiom) and [configure_otel_logging_axiom](/workflows/observability/logging#axiom).

### Jaeger

<img src="/assets/workflows/observability/tilebox-jaeger.png" alt="Tilebox works well with Jaeger" />

Another popular option is [Jaeger](https://jaegertracing.io/), a popular distributed tracing system. You can use the [OpenTelemetry Collector](https://opentelemetry.io/docs/collector/) to collect telemetry data from Tilebox Workflows and export it to Jaeger.

An all-in-one Jaeger environment can be spun up using Docker:

```bash
docker run --rm --name jaeger \
-p 5778:5778 \
-p 16686:16686 \
-p 4318:4318 \
jaegertracing/jaeger:2.9.0
```

You can then configure Tilebox to export traces to Jaeger using `configure_otel_tracing`.

<CodeGroup>
```python Python
from tilebox.workflows import Client
from tilebox.workflows.observability.tracing import configure_otel_tracing

# your own workflow:
from my_workflow import MyTask

def main():
configure_otel_tracing(
# export traces to the local Jaeger instance
endpoint="http://localhost:4318",
)

# the following task runner generates traces for executed tasks and
# exports trace and span data to the specified endpoint
client = Client()
runner = client.runner(tasks=[MyTask])
runner.run_forever()

if __name__ == "__main__":
main()
```
```go Go
package main

import (
"context"
"log/slog"

"github.com/tilebox/tilebox-go/examples/workflows/opentelemetry"
"github.com/tilebox/tilebox-go/observability"
"github.com/tilebox/tilebox-go/observability/tracer"
"github.com/tilebox/tilebox-go/workflows/v1"
"go.opentelemetry.io/otel"
)

// specify a service name and version to identify the instrumenting application in traces and logs
var service = &observability.Service{Name: "task-runner", Version: "dev"}

func main() {
ctx := context.Background()

endpoint := "http://localhost:4318"

// Setup an OpenTelemetry trace span processor, exporting traces and spans to an OTEL compatible trace endpoint
tileboxTracerProvider, shutdown, err := tracer.NewOtelProvider(ctx, service, tracer.WithEndpointURL(endpoint))
defer shutdown(ctx)
if err != nil {
slog.Error("failed to set up otel span processor", slog.Any("error", err))
return
}
otel.SetTracerProvider(tileboxTracerProvider) // set the tilebox tracer provider as the global OTEL tracer provider

client := workflows.NewClient()

taskRunner, err := client.NewTaskRunner(ctx)
if err != nil {
slog.Error("failed to create task runner", slog.Any("error", err))
return
}

err = taskRunner.RegisterTasks(&MyTask{})
if err != nil {
slog.Error("failed to register tasks", slog.Any("error", err))
return
}

taskRunner.RunForever(ctx)
}
```
</CodeGroup>

Additionally, any other OpenTelemetry-compatible backend, such as OpenTelemetry Collector or Jaeger, can be used to collect telemetry data generated by Tilebox Workflows.
The generated workflow traces can then be viewed in the Jaeger UI at [http://localhost:16686](http://localhost:16686).