Observability

Every function execution produces one OpenTelemetry trace: a root span for the invocation, child spans for every HTTP call, database query, and KV operation, and structured ctx.log records correlated to the same trace. Instrumentation is built into the context wrappers, so there’s nothing to configure. Read the results with helix logs or in the Helix dashboard.

Built on OpenTelemetry

Helix observability is standard OpenTelemetry (OTel) end to end. Each execution emits two distinct signals, serialized over OTLP:

• Traces (spans). Each execution is one trace. The function invocation is the root span; each HTTP call, DB query, and KV operation is a child span with typed attributes. Spans carry timing, status, and structured attributes, which power the timeline, latency views, and searching by any operation’s properties.
• Logs (log records). ctx.log emits OTel log records: severity, body, and attributes, correlated to the currently active span through its trace and span IDs. A log record is not a span; the two signals are stored and queried separately, matching how OTel backends model them.

Both signals carry the same resource attributes identifying the tenant and execution: organizationId, workspaceId, projectId, executionId, functionPath, and method. These are stamped server-side at ingest from the verified execution token; nothing the function process claims about its identity is trusted.

Because the wire format is OTLP, telemetry can be exported to any OTel-compatible backend (Honeycomb, Datadog, Grafana Tempo, your own Collector), and standard trace-context propagation lets an execution participate in a larger distributed trace across other functions and external systems.

Write logs with ctx.log

ctx.log is the primary logging interface. It produces structured, execution-scoped log records that are automatically correlated with the execution’s trace.

interface ContextLogger {
  info(message: string, metadata?: Record<string, unknown>): void;
  warn(message: string, metadata?: Record<string, unknown>): void;
  error(message: string, metadata?: Record<string, unknown>): void;
  debug(message: string, metadata?: Record<string, unknown>): void;
}

// functions/api/v1/sync.post.ts
import { defineFunction } from '@trayai/helix-sdk';
import { z } from 'zod';
import { contactsTable } from '../../../db/schema';

export const input = z.object({
  userId: z.string(),
});

export default defineFunction<typeof input>(async (ctx) => {
  ctx.log.info('Starting sync', { userId: ctx.input.userId });

  const contacts = await ctx.http
    .authed('salesforce_prod')
    .get('{{instance_url}}/services/data/v59.0/query', {
      searchParams: { q: 'SELECT Id, Name FROM Contact LIMIT 10' },
    })
    .json();

  ctx.log.info('Fetched contacts', { count: contacts.records.length });

  for (const contact of contacts.records) {
    await ctx.db().insert(contactsTable).values({
      external_id: contact.Id,
      name: contact.Name,
    }).onConflictDoUpdate({ target: contactsTable.external_id, set: { name: contact.Name } });
  }

  ctx.log.warn('Duplicate check skipped (feature flag disabled)');
  return { synced: contacts.records.length };
});

Each call emits one OTel log record carrying:

• Timestamp (observed time)
• Severity text and number, mapped from the level
• Body, the message string
• Attributes, your metadata object
• Trace and span IDs of the active execution span, for correlation
• Resource attributes (executionId, projectId, workspaceId, organizationId, functionPath, method), stamped at ingest

Metadata is carried as a single serialized JSON attribute rather than exploded into one attribute per key, which keeps log-record attribute cardinality bounded. In local dev, ctx.log also pretty-prints to the terminal for immediate visibility.

The dashboard interleaves the trace’s spans and its correlated log records into one time-ordered execution timeline:

exec_789xyz | 2025-03-19T10:30:00.123Z | POST /api/v1/sync (trace root: "function")

10:30:00.123  LOG  INFO   Starting sync { userId: "user_abc" }
10:30:00.145  SPAN HTTP → GET https://mycompany.my.salesforce.com/services/data/v59.0/query
                          auth: salesforce_prod
10:30:00.412  SPAN HTTP ← 200 OK (267ms)
10:30:00.413  LOG  INFO   Fetched contacts { count: 10 }
10:30:00.415  SPAN DB   → INSERT INTO contacts ... ON CONFLICT DO UPDATE
10:30:00.418  SPAN DB   ← OK (3ms)
10:30:00.450  LOG  WARN   Duplicate check skipped (feature flag disabled)
10:30:00.451  ←    200 OK { synced: 10 } (328ms total)

Full API reference: ctx.log.

Auto-instrumented spans

HTTP calls, database queries, and KV operations are emitted as child spans of the execution’s root span, each with typed attributes the dashboard (and any OTel backend) can render and filter.

HTTP calls. Authenticated calls via ctx.http.authed(...) capture the request method, URL (templates resolved), headers (credentials redacted), and body; the response status, headers, and body (truncated if large); the duration; and the auth alias used, plus whether a token refresh occurred. Unauthenticated calls capture method, URL, headers, response status, and duration.

Database queries. Every Drizzle operation via ctx.db() produces a span with the SQL query in parameterized form (no actual values, for security), the parameter count, the row count returned, the duration, and whether it ran inside a transaction.

KV operations. Every ctx.kv() operation produces a span with the operation type (get, set, delete, incr, list), the key (or prefix for list), the scope (project, workspace, or execution), whether a TTL was set, and the duration.

The root "function" span also carries roll-up attributes, such as whether the execution made any auth, external, or DB call and its overall error status, so the executions list can filter with a single term query instead of scanning child spans.

Dev and production transport

You write the same code in both environments; the runtime picks the transport.

• Local dev (helix dev). ctx.log pretty-prints to stdout: colored level, readable timestamp, message, and metadata inline. Spans print similarly. No OTLP export, no ingest endpoint.
• Production (Lambda). The SDK exports OTLP/protobuf through batching processors: spans to {ingestUrl}/v1/traces, log records to {ingestUrl}/v1/logs. Each export is authenticated with the short-lived Helix execution token in the X-Helix-Project-Execution-Token header, resolved per invocation. Both processors flush at the end of every execution; on a crash, whatever was buffered, plus the error, is flushed before the process is frozen.

The ingest endpoint verifies the token, stamps the tenant and resource identity from the verified claims (overwriting anything the client set), and forwards the payload to the trace and log stores. Because the payload is plain OTLP, the same stream can also point at any OTel-compatible backend.

console.log capture

console.log, console.warn, console.error, and console.debug are captured best-effort alongside ctx.log, so functions that haven’t adopted ctx.log still get some observability without refactoring. The Lambda runtime installs a lightweight stdout formatter (not a global console patch) that prefixes each line with the execution ID, and the log aggregation service parses those lines back into the right execution timeline:

[exec_789xyz] INFO Starting sync { userId: "user_abc" }

Known limitations, which is why ctx.log is the reliable interface:

• No structured attributes. console.log('foo', { bar: 1 }) produces a string; ctx.log.info('foo', { bar: 1 }) produces a log record with queryable attributes.
• Third-party library output. Captured with the execution-ID prefix, but may lack useful structure.
• Serialization. Complex objects may not serialize cleanly; ctx.log serializes reliably.
• Performance. Parsing stdout lines is slower than direct ctx.log emission; high-volume logging should use ctx.log.

Error tracking

Unhandled errors are captured automatically: the root span’s status is set to error and an error-level log record is emitted, both correlated on the same trace. The record includes the full stack trace with source-mapped file paths, and the timeline shows everything up to the failure plus the request that triggered it:

exec_abc123 | 2025-03-19T10:30:00.123Z | GET /api/v1/users/:userId (trace status: ERROR)

10:30:00.123  LOG  INFO   Fetching user
10:30:00.200  SPAN HTTP → GET https://mycompany.my.salesforce.com/...
10:30:00.450  SPAN HTTP ← 401 Unauthorized (250ms)
10:30:00.451  LOG  ERROR  TypeError: Cannot read property 'records' of undefined
                          at handler (functions/api/v1/users/[userId].get.ts:15:34)
10:30:00.452  ←    500 Internal Server Error (329ms total)

Read logs

# Recent executions across all functions
helix logs

# Follow mode (like tail -f)
helix logs --follow

# Filter by function
helix logs --function api/v1/users

# Filter by level
helix logs --level error

# Specific execution
helix logs --execution exec_789xyz

# Time range
helix logs --since 1h
helix logs --since "2025-03-19T10:00:00Z"

# AI gateway request logs
helix logs --type ai
helix logs --type ai --filter model=gpt-4o

The Helix dashboard shows the same data as interleaved execution timelines, searchable by any operation’s properties. helix logs is not in the shipped CLI; the CLI reference lists every command that is.