Skip to content
Opswald Docs

Proxy Overview

Proxy Overview

Debug your AI agents without changing a single line of code. The Opswald proxy sits between your application and LLM providers, automatically capturing every API call as structured traces.

How It Works

Your App → Opswald Proxy → OpenAI/Anthropic → Response → Proxy → Your App
            ↓
        Opswald Dashboard

The proxy:

  1. Intercepts all HTTP requests to LLM providers
  2. Captures request/response data as spans
  3. Forwards requests to the original provider unchanged
  4. Sends trace data to Opswald for debugging

Zero Code Changes

Just change your base URL:

# Before
import openai
client = openai.OpenAI(base_url="https://api.openai.com/v1")


# After
import openai
client = openai.OpenAI(base_url="https://proxy.opswald.com/openai")
// Before
import OpenAI from 'openai';
const client = new OpenAI({ baseURL: "https://api.openai.com/v1" });


// After
import OpenAI from 'openai';
const client = new OpenAI({ baseURL: "https://proxy.opswald.com/openai" });

Your API key stays the same. Your code stays the same. Only the URL changes.

Supported Providers

ProviderProxy URLOriginal URL
OpenAIhttps://proxy.opswald.com/openaihttps://api.openai.com/v1
Anthropichttps://proxy.opswald.com/anthropichttps://api.anthropic.com
Google AIhttps://proxy.opswald.com/googlehttps://generativelanguage.googleapis.com

What Gets Captured

Every API call becomes a span with:

Request Data

  • Model used (gpt-4o, claude-3-sonnet, etc.)
  • Input tokens (cost calculated server-side during ingestion)
  • Parameters (temperature, max_tokens, etc.)
  • Messages (with content filtering options)

Response Data

  • Generated content (with filtering options)
  • Output tokens (cost calculated server-side during ingestion)
  • Response time and latency
  • Finish reason (completed, length, etc.)

Metadata

  • Timestamp and duration
  • User ID (if provided in headers)
  • Session ID (if provided in headers)
  • Error details (for failed requests)

Quick Start

1. Get Your Proxy Token

Terminal window
curl -X POST https://api.opswald.com/v1/proxy/tokens \
  -H "Authorization: Bearer your-opswald-api-key" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "my-app-proxy",
    "description": "Proxy token for my AI application"
  }'

2. Configure Your Application

Add these headers to your LLM requests:

import openai


client = openai.OpenAI(
    base_url="https://proxy.opswald.com/openai",
    default_headers={
        "X-Opswald-Key": "your-proxy-token",
        "X-Opswald-Session": "user-session-123",  # Optional
        "X-Opswald-User": "user-456"              # Optional
    }
)

3. Make Requests Normally

# This call is now automatically traced
response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Hello!"}]
)

4. View Traces in Dashboard

Go to app.opswald.com to see your traces appear in real-time.

Content Filtering

Control what content gets captured for privacy:

client = openai.OpenAI(
    base_url="https://proxy.opswald.com/openai",
    default_headers={
        "X-Opswald-Key": "your-proxy-token",
        "X-Opswald-Filter-Input": "hash",      # hash|truncate|omit|full
        "X-Opswald-Filter-Output": "truncate"  # hash|truncate|omit|full
    }
)

Filter Options

OptionInput BehaviorOutput Behavior
fullCapture complete contentCapture complete content
truncateFirst 200 chars + ”…”First 200 chars + ”…”
hashSHA-256 hash onlySHA-256 hash only
omitNo content capturedNo content captured

Session and User Tracking

Group related calls and identify users:

# Track user sessions
client = openai.OpenAI(
    base_url="https://proxy.opswald.com/openai",
    default_headers={
        "X-Opswald-Key": "your-proxy-token",
        "X-Opswald-Session": f"chat-{session_id}",
        "X-Opswald-User": f"user-{user_id}",
        "X-Opswald-Trace": "customer-support-flow"  # Groups spans into traces
    }
)


# All subsequent requests in this conversation will be grouped
response1 = client.chat.completions.create(...)  # Span 1
response2 = client.chat.completions.create(...)  # Span 2
response3 = client.chat.completions.create(...)  # Span 3

Error Handling

The proxy handles errors gracefully:

Provider Errors

If OpenAI/Anthropic returns an error, the proxy:

  1. Captures the error in a span
  2. Forwards the exact error to your app
  3. Preserves all error details and status codes

Proxy Errors

If the proxy itself fails:

  1. Fallback requests go directly to the provider
  2. No interruption to your application
  3. Error logged but doesn’t break your workflow

Custom Metadata

Add custom context to your traces:

client = openai.OpenAI(
    base_url="https://proxy.opswald.com/openai",
    default_headers={
        "X-Opswald-Key": "your-proxy-token",
        "X-Opswald-Meta-Environment": "production",
        "X-Opswald-Meta-Version": "v2.1.0",
        "X-Opswald-Meta-Feature": "customer-chat",
        "X-Opswald-Meta-User-Tier": "enterprise"
    }
)

Custom metadata appears in the dashboard for filtering and analysis.

Streaming Support

The proxy fully supports streaming responses:

# Streaming works transparently
stream = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Tell me a story"}],
    stream=True
)


for chunk in stream:
    print(chunk.choices[0].delta.content)
    # Each chunk is captured, final span shows complete response

Framework Integration

LangChain

from langchain.llms import OpenAI


llm = OpenAI(
    openai_api_base="https://proxy.opswald.com/openai",
    openai_api_key="your-openai-key",  # Your real OpenAI key
    headers={
        "X-Opswald-Key": "your-proxy-token",
        "X-Opswald-Session": "langchain-session"
    }
)


# All LangChain LLM calls now traced automatically
response = llm("What is the capital of France?")

Haystack

from haystack.nodes import PromptNode


prompt_node = PromptNode(
    model_name_or_path="gpt-4o",
    api_key="your-openai-key",
    api_base="https://proxy.opswald.com/openai",
    default_headers={
        "X-Opswald-Key": "your-proxy-token"
    }
)

Guidance

import guidance


# Configure guidance to use proxy
guidance.llm = guidance.llms.OpenAI(
    api_base="https://proxy.opswald.com/openai",
    api_key="your-openai-key",
    headers={"X-Opswald-Key": "your-proxy-token"}
)

Self-Hosted Proxy

Deploy your own proxy instance for maximum control:

Terminal window
# Docker
docker run -d -p 8080:8080 \
  -e OPSWALD_API_KEY=your-key \
  -e OPSWALD_API_URL=https://api.opswald.com \
  opswald/proxy:latest


# Your base URL becomes
# http://localhost:8080/openai/v1

See Configuration for full deployment options.

Benefits

For Development

  • See every LLM call without code changes
  • Debug conversation flows across multiple requests
  • Monitor token usage and costs in real-time
  • Identify bottlenecks with request timing

For Production

  • Track user interactions with session grouping
  • Monitor error rates and failure patterns
  • Review LLM usage patterns for debugging
  • Performance monitoring with latency metrics

For Debugging

  • Zero setup time - just change the URL
  • Complete conversation history in one view
  • Error reproduction with exact request data
  • A/B testing different models or parameters

The proxy gives you instant visibility into any LLM-powered application, making it the fastest way to start debugging AI agents.