LangChain Integration - Helicone OSS LLM Observability

Introduction

LangChain is a popular open-source framework for building applications with large language models across Python, TypeScript, and other languages. By integrating Helicone AI Gateway with LangChain, you can:

Route to different models & providers with automatic failover through a single endpoint
Unified billing with pass-through billing or bring your own keys
Monitor all requests with automatic cost tracking in one dashboard
Stream responses with full observability for real-time applications

This integration requires only two changes to your existing LangChain code - updating the base URL and API key.

Integration Steps

Create an account + Generate an API Key

You’ll also need to configure your provider API keys (OpenAI, Anthropic, etc.) at Helicone Providers for BYOK (Bring Your Own Keys).

Set environment variables

# Your Helicone API key
export HELICONE_API_KEY=<your-helicone-api-key>

Create a .env file in your project:

HELICONE_API_KEY=sk-helicone-...

Install LangChain packages

npm install @langchain/openai @langchain/core dotenv
# or
yarn add @langchain/openai @langchain/core dotenv

pip install langchain-openai langchain-core python-dotenv

Configure LangChain with Helicone AI Gateway

import { ChatOpenAI } from "@langchain/openai";
import { HumanMessage, SystemMessage } from "@langchain/core/messages";
import dotenv from 'dotenv';

dotenv.config();

// Initialize ChatOpenAI with Helicone AI Gateway
const chat = new ChatOpenAI({
    model: 'gpt-4.1-mini',  // 100+ models supported
    apiKey: process.env.HELICONE_API_KEY,
    configuration: {
        baseURL: "https://ai-gateway.helicone.ai/v1",
        defaultHeaders: {
            // Optional: Add custom tracking headers
            "Helicone-Session-Id": "my-session",
            "Helicone-User-Id": "user-123",
            "Helicone-Property-Environment": "production",
        },
    },
});

import os
from langchain_openai import ChatOpenAI
from langchain_core.messages import HumanMessage, SystemMessage
from dotenv import load_dotenv

load_dotenv()

# Initialize ChatOpenAI with Helicone AI Gateway
chat = ChatOpenAI(
    model='gpt-4.1-mini',  # 100+ models supported
    api_key=os.getenv('HELICONE_API_KEY'),
    base_url="https://ai-gateway.helicone.ai/v1",
    default_headers={
        # Optional: Add custom tracking headers
        'Helicone-Session-Id': 'my-session',
        'Helicone-User-Id': 'user-123',
        'Helicone-Property-Environment': 'production',
    },
)

The only changes from a standard LangChain setup are the apiKey, baseURL (or base_url in Python), and optional tracking headers. Everything else stays the same!

Use LangChain normally

Your existing LangChain code continues to work without any changes:

// Simple completion
const response = await chat.invoke([
    new SystemMessage("You are a helpful assistant."),
    new HumanMessage("What is the capital of France?"),
]);

console.log(response.content);

# Simple completion
messages = [
    SystemMessage(content="You are a helpful assistant."),
    HumanMessage(content="What is the capital of France?"),
]

response = chat.invoke(messages)
print(response.content)

Request/response bodies
Latency metrics
Token usage and costs
Model performance analytics
Error tracking
Session tracking

While you’re here, why not give us a star on GitHub? It helps us a lot!

Migration Example

Here’s what migrating an existing LangChain application looks like:

Before (Direct OpenAI)

import { ChatOpenAI } from "@langchain/openai";

const chat = new ChatOpenAI({
    model: 'gpt-4o-mini',
    apiKey: process.env.OPENAI_API_KEY,
});

from langchain_openai import ChatOpenAI

chat = ChatOpenAI(
    model='gpt-4o-mini',
    api_key=os.getenv('OPENAI_API_KEY'),
)

After (Helicone AI Gateway)

import { ChatOpenAI } from "@langchain/openai";

const chat = new ChatOpenAI({
    model: 'gpt-4.1-mini',                      // 100+ models supported
    apiKey: process.env.HELICONE_API_KEY,      // Your Helicone API key
    configuration: {
        baseURL: "https://ai-gateway.helicone.ai/v1"  // Add this!
    },
});

from langchain_openai import ChatOpenAI

chat = ChatOpenAI(
    model='gpt-4.1-mini',                      # 100+ models supported
    api_key=os.getenv('HELICONE_API_KEY'),    # Your Helicone API key
    base_url="https://ai-gateway.helicone.ai/v1"  # Add this!
)

That’s it! Just two changes and you’re routing through Helicone’s AI Gateway.

Complete Working Examples

Basic Example

import { ChatOpenAI } from "@langchain/openai";
import { HumanMessage, SystemMessage } from "@langchain/core/messages";
import dotenv from 'dotenv';

dotenv.config();

const chat = new ChatOpenAI({
    model: 'gpt-4.1-mini',  // 100+ models supported
    apiKey: process.env.HELICONE_API_KEY,
    configuration: {
        baseURL: "https://ai-gateway.helicone.ai/v1",
        defaultHeaders: {
            "Helicone-Session-Id": "langchain-example",
            "Helicone-User-Id": "demo-user",
        },
    },
});

async function main() {
    console.log('🦜 Starting LangChain + Helicone AI Gateway example...\n');

    const response = await chat.invoke([
        new SystemMessage("You are a helpful assistant."),
        new HumanMessage("Tell me a joke about programming."),
    ]);

    console.log('🤖 Assistant response:');
    console.log(response.content);
    console.log('\n✅ Completed successfully!');
}

main().catch(console.error);

import os
from langchain_openai import ChatOpenAI
from langchain_core.messages import HumanMessage, SystemMessage
from dotenv import load_dotenv

load_dotenv()

chat = ChatOpenAI(
    model='gpt-4.1-mini',  # 100+ models supported
    api_key=os.getenv('HELICONE_API_KEY'),
    base_url="https://ai-gateway.helicone.ai/v1",
    default_headers={
        'Helicone-Session-Id': 'langchain-example',
        'Helicone-User-Id': 'demo-user',
    },
)

def main():
    print('🐍 Starting LangChain + Helicone AI Gateway example...\n')

    messages = [
        SystemMessage(content="You are a helpful assistant."),
        HumanMessage(content="Tell me a joke about Python programming."),
    ]

    response = chat.invoke(messages)

    print('🤖 Assistant response:')
    print(response.content)
    print('\n✅ Completed successfully!')

if __name__ == "__main__":
    main()

Streaming Example

async function streamingExample() {
    console.log('\n🌊 Streaming example...\n');

    const stream = await chat.stream([
        new SystemMessage("You are a helpful assistant."),
        new HumanMessage("Write a short story about a robot learning to code."),
    ]);

    console.log('🤖 Assistant (streaming):');
    for await (const chunk of stream) {
        process.stdout.write(chunk.content as string);
    }
    console.log('\n\n✅ Streaming completed!');
}

streamingExample().catch(console.error);

def streaming_example():
    print('\n🌊 Streaming example...\n')

    messages = [
        SystemMessage(content="You are a helpful assistant."),
        HumanMessage(content="Write a short story about a robot learning to code."),
    ]

    print('🤖 Assistant (streaming):')
    for chunk in chat.stream(messages):
        print(chunk.content, end='', flush=True)

    print('\n\n✅ Streaming completed!')

streaming_example()

Multiple Models Example

async function testMultipleModels() {
    console.log('🚀 Testing multiple models through Helicone AI Gateway\n');

    const models = [
        { id: 'gpt-4.1-mini', name: 'OpenAI GPT-4.1 Mini' },
        { id: 'claude-opus-4-1', name: 'Anthropic Claude Opus 4.1' },
        { id: 'gemini-2.5-flash-lite', name: 'Google Gemini 2.5 Flash Lite' },
    ];

    for (const model of models) {
        try {
            const chat = new ChatOpenAI({
                model: model.id,
                apiKey: process.env.HELICONE_API_KEY,
                configuration: {
                    baseURL: "https://ai-gateway.helicone.ai/v1",
                },
            });

            console.log(`🤖 Testing ${model.name}... `);
            const response = await chat.invoke([
                new HumanMessage("Say hello in one sentence."),
            ]);
            console.log(`   Response: ${response.content}\n`);
        } catch (error) {
            console.error(`   Error: ${error}\n`);
        }
    }

    console.log('✅ All models tested!');
    console.log('🔍 Check your dashboard: https://us.helicone.ai/dashboard');
}

testMultipleModels().catch(console.error);

def test_multiple_models():
    print('🚀 Testing multiple models through Helicone AI Gateway\n')

    models = [
        {'id': 'gpt-4.1-mini', 'name': 'OpenAI GPT-4.1 Mini'},
        {'id': 'claude-opus-4-1', 'name': 'Anthropic Claude Opus 4.1'},
        {'id': 'gemini-2.5-flash-lite', 'name': 'Google Gemini 2.5 Flash Lite'},
    ]

    for model in models:
        try:
            chat = ChatOpenAI(
                model=model['id'],
                api_key=os.getenv('HELICONE_API_KEY'),
                base_url="https://ai-gateway.helicone.ai/v1",
            )

            print(f"🤖 Testing {model['name']}... ")
            response = chat.invoke([
                HumanMessage(content="Say hello in one sentence."),
            ])
            print(f"   Response: {response.content}\n")
        except Exception as error:
            print(f"   Error: {error}\n")

    print('✅ All models tested!')
    print('🔍 Check your dashboard: https://us.helicone.ai/dashboard')

test_multiple_models()

Batch Processing Example (Python)

Python

def batch_example():
    print('\n📦 Batch processing example...\n')

    message_batches = [
        [HumanMessage(content="What is Python?")],
        [HumanMessage(content="What is JavaScript?")],
        [HumanMessage(content="What is TypeScript?")],
    ]

    responses = chat.batch(message_batches)

    print('🤖 Batch responses:')
    for i, response in enumerate(responses, 1):
        print(f'\nResponse {i}: {response.content}')

    print('\n✅ Batch processing completed!')

batch_example()

Helicone Prompts Integration

You can use Helicone Prompts for centralized prompt management and versioning by passing parameters through modelKwargs:

const chat = new ChatOpenAI({
    model: 'gpt-4.1-mini',
    apiKey: process.env.HELICONE_API_KEY,
    modelKwargs: {
        prompt_id: 'customer-support-prompt',
        version_id: 'version-uuid',
        environment: 'production',
        inputs: { customer_name: 'John', issue_type: 'billing' },
    },
    configuration: {
        baseURL: "https://ai-gateway.helicone.ai/v1",
    },
});

chat = ChatOpenAI(
    model='gpt-4.1-mini',
    api_key=os.getenv('HELICONE_API_KEY'),
    base_url="https://ai-gateway.helicone.ai/v1",
    model_kwargs={
        'prompt_id': 'customer-support-prompt',
        'version_id': 'version-uuid',
        'environment': 'production',
        'inputs': {'customer_name': 'John', 'issue_type': 'billing'},
    },
)

All prompt parameters (prompt_id, version_id, environment, inputs) are optional. Learn more about Prompts with AI Gateway.

Custom Headers and Properties

You can add custom properties to track and filter your requests:

const chat = new ChatOpenAI({
    model: 'gpt-4.1-mini',
    apiKey: process.env.HELICONE_API_KEY,
    configuration: {
        baseURL: "https://ai-gateway.helicone.ai/v1",
        defaultHeaders: {
            // Session tracking
            "Helicone-Session-Id": "session-abc-123",
            "Helicone-Session-Name": "Customer Support Chat",
            "Helicone-Session-Path": "/support/chat/456",

            // User tracking
            "Helicone-User-Id": "user-789",

            // Custom properties for filtering
            "Helicone-Property-Environment": "production",
            "Helicone-Property-App-Version": "2.1.0",
            "Helicone-Property-Feature": "customer-support",

            // Rate limiting (optional)
            "Helicone-Rate-Limit-Policy": "basic-100",
        },
    },
});

chat = ChatOpenAI(
    model='gpt-4.1-mini',
    api_key=os.getenv('HELICONE_API_KEY'),
    base_url="https://ai-gateway.helicone.ai/v1",
    default_headers={
        # Session tracking
        'Helicone-Session-Id': 'session-abc-123',
        'Helicone-Session-Name': 'Customer Support Chat',
        'Helicone-Session-Path': '/support/chat/456',

        # User tracking
        'Helicone-User-Id': 'user-789',

        # Custom properties for filtering
        'Helicone-Property-Environment': 'production',
        'Helicone-Property-App-Version': '2.1.0',
        'Helicone-Property-Feature': 'customer-support',

        # Rate limiting (optional)
        'Helicone-Rate-Limit-Policy': 'basic-100',
    },
)

Looking for a framework or tool not listed here? Request it here!

AI Gateway Overview

Learn about Helicone’s AI Gateway features and capabilities

Provider Routing

Configure intelligent routing and automatic failover

Model Registry

Browse all available models and providers

Prompt Management

Version and manage prompts with Helicone Prompts

Custom Properties

Add metadata to track and filter your requests

Sessions

Track multi-turn conversations and user sessions

Rate Limiting

Configure rate limits for your applications

​Introduction

​Integration Steps

​Migration Example

​Before (Direct OpenAI)

​After (Helicone AI Gateway)

​Complete Working Examples

​Basic Example

​Streaming Example

​Multiple Models Example

​Batch Processing Example (Python)

​Helicone Prompts Integration

​Custom Headers and Properties

​Related Documentation

AI Gateway Overview

Provider Routing

Model Registry

Prompt Management

Custom Properties

Sessions

Rate Limiting

Introduction

Integration Steps

Migration Example

Before (Direct OpenAI)

After (Helicone AI Gateway)

Complete Working Examples

Basic Example

Streaming Example

Multiple Models Example

Batch Processing Example (Python)

Helicone Prompts Integration

Custom Headers and Properties

Related Documentation