Skip to content

AGENTS.md

Latest commit

 

History

History
109 lines (83 loc) · 4.74 KB

AGENTS.md

File metadata and controls

109 lines (83 loc) · 4.74 KB
Raw

Agent Guide

Context for AI coding assistants working on this codebase.

Project Overview

This is a Cloudflare Worker that acts as an optimizing proxy for OTLP (OpenTelemetry Protocol) JSON payloads. It receives logs and traces, deduplicates redundant Resource/Scope declarations, and forwards the optimized payload to configured destinations.

Why it exists: Upstream batch processors often emit payloads with repeated identical Resource and Scope sections for each record. This proxy consolidates them to reduce payload size by 60-70%.

Architecture

src/
├── index.ts              # Worker entry point, routing
├── config.ts             # Configuration loading from env
├── types/otlp.ts         # OTLP type definitions
├── middleware/auth.ts    # Bearer token authentication
├── handlers/otlp.ts      # Request handlers for /v1/logs and /v1/traces
├── otlp/
│   ├── parsing.ts        # Decompression and JSON parsing
│   ├── optimizer.ts      # Core grouping/deduplication logic
│   ├── equality.ts       # Deep comparison for Resource/Scope
│   └── response.ts       # Response formatting utilities
└── forwarding/client.ts  # Outgoing request handling (gzip, headers)

Key Patterns

Separate functions over parameterized ones

The codebase uses separate functions for logs vs traces rather than a single parameterized function. For example:

  • parseOtlpLogsRequest() and parseOtlpTracesRequest() (not parseOtlpRequest(type))
  • optimizeLogs() and optimizeTraces() (not optimize(type))
  • handleLogs() and handleTraces() (not handle(endpoint))

Error handling

  • Custom error classes: ParseError, ConfigError, AuthError
  • Errors include HTTP status codes for proper response generation
  • Header values are sanitized in error messages to prevent leaking secrets

Testing

  • Tests are co-located with source in test/ directory mirroring src/ structure
  • Uses Vitest with @cloudflare/vitest-pool-workers for Workers runtime
  • 304 tests covering unit, integration, and E2E scenarios
  • Run tests: npm test -- --run

Important Files

File Purpose
src/otlp/optimizer.ts Core optimization logic - where deduplication happens
src/otlp/equality.ts Hash functions for comparing Resources and Scopes
src/config.ts Env interface defines all environment variables
wrangler.jsonc Cloudflare Worker configuration
plan.md Original implementation plan with design decisions

Common Tasks

Adding a new environment variable

  1. Add to Env interface in src/config.ts
  2. Add validation in loadConfig() if required
  3. Update wrangler.jsonc with example/documentation
  4. Update README.md configuration reference

Modifying optimization logic

The optimizer in src/otlp/optimizer.ts uses Maps keyed by hashed Resource/Scope values:

  • hashResource() and hashScope() in equality.ts create stable identity keys
  • Records with identical keys are consolidated under a single Resource/Scope

Adding a new endpoint

  1. Add handler in src/handlers/
  2. Add route in src/index.ts
  3. Add tests in test/handlers/ and test/e2e/

Known Issues

DecompressionStream unhandled rejections

The workerd DecompressionStream throws unhandled promise rejections internally when given invalid compressed data, even when our code catches the error. We removed tests for invalid gzip/deflate data to avoid spurious test failures. The error handling works correctly in production.

OTLP validation is minimal

We validate top-level structure (resourceLogs/resourceSpans arrays) but don't deeply validate OTLP schema. Invalid nested structures pass through and may cause issues at the destination.

Environment Variables

Variable Required Secret Description
AUTH_TOKEN Yes Yes Bearer token for incoming requests
LOGS_DESTINATION_URL Yes No URL to forward logs
TRACES_DESTINATION_URL Yes No URL to forward traces
LOGS_DESTINATION_HEADERS No Maybe JSON headers for logs (secret if contains API keys)
TRACES_DESTINATION_HEADERS No Maybe JSON headers for traces (secret if contains API keys)

Development Commands

npm test -- --run          # Run all tests
npm test -- --watch        # Watch mode
wrangler dev               # Local dev server (needs .dev.vars)
wrangler deploy            # Deploy to Cloudflare
wrangler secret put NAME   # Set a secret

Design Decisions

Key decisions:

  1. JSON only - No protobuf support needed for this use case
  2. Pass-through errors - Destination errors returned to client for retry handling
  3. Always gzip outgoing - All forwarded requests are gzip compressed