DSPy

Introduction

DSPy is a declarative framework for building modular AI software with structured code instead of brittle prompts, offering algorithms that compile AI programs into effective prompts and weights for language models across classifiers, RAG pipelines, and agent loops.

Integration Steps

Create a .env file in your project.

HELICONE_API_KEY=sk-helicone-...

Python

pip install dspy

Python

import dspy
import os
from dotenv import load_dotenv

load_dotenv()

# Configure DSPy to use Helicone AI Gateway
lm = dspy.LM(
  'gpt-4o-mini',  # or any other model from the Helicone model registry
  api_key=os.getenv('HELICONE_API_KEY'),
  api_base='https://ai-gateway.helicone.ai/'
)

dspy.configure(lm=lm)

print(lm("Hello, world!"))

While you’re here, why not give us a star on GitHub? It helps us a lot!

Complete Working Examples

Basic Chain of Thought

Python

import dspy
import os
from dotenv import load_dotenv

load_dotenv()

# Configure Helicone AI Gateway
lm = dspy.LM(
    'gpt-4o-mini',
    api_key=os.getenv('HELICONE_API_KEY'),
    api_base='https://ai-gateway.helicone.ai/v1'
)
dspy.configure(lm=lm)

# Define a module
qa = dspy.ChainOfThought('question -> answer')

# Run inference
response = qa(question="How many floors are in the castle David Gregory inherited?")

print('Answer:', response.answer)
print('Reasoning:', response.reasoning)

Custom Generation Configuration

Configure temperature, max_tokens, and other parameters:

Python

import dspy
import os
from dotenv import load_dotenv

load_dotenv()

# Configure with custom generation parameters
lm = dspy.LM(
    'gpt-4o-mini',
    api_key=os.getenv('HELICONE_API_KEY'),
    api_base='https://ai-gateway.helicone.ai/v1',
    temperature=0.9,
    max_tokens=2000
)
dspy.configure(lm=lm)

# Use with any DSPy module
predict = dspy.Predict("question -> creative_answer")
response = predict(question="Write a creative story about AI")
print(response.creative_answer)

Tracking with Custom Properties

Add custom properties to track and filter your requests in the Helicone dashboard:

Python

import dspy
import os
from dotenv import load_dotenv

load_dotenv()

# Configure with custom Helicone headers
lm = dspy.LM(
    'gpt-4o-mini',
    api_key=os.getenv('HELICONE_API_KEY'),
    api_base='https://ai-gateway.helicone.ai/v1',
    extra_headers={
        # Session tracking
        'Helicone-Session-Id': 'dspy-example-session',
        'Helicone-Session-Name': 'Question Answering',

        # User tracking
        'Helicone-User-Id': 'user-123',

        # Custom properties for filtering
        'Helicone-Property-Environment': 'production',
        'Helicone-Property-Module': 'chain-of-thought',
        'Helicone-Property-Version': '1.0.0'
    }
)
dspy.configure(lm=lm)

# Use normally
qa = dspy.ChainOfThought('question -> answer')
response = qa(question="What is DSPy?")
print(response.answer)

Helicone Prompts Integration

Use Helicone Prompts for centralized prompt management with DSPy signatures:

Python

import dspy
import os
from dotenv import load_dotenv

load_dotenv()

# Configure with prompt parameters
lm = dspy.LM(
    'gpt-4o-mini',
    api_key=os.getenv('HELICONE_API_KEY'),
    api_base='https://ai-gateway.helicone.ai/v1',
    extra_body={
        'prompt_id': 'customer-support-prompt-id',
        'version_id': 'version-uuid',
        'environment': 'production',
        'inputs': {
            'customer_name': 'Sarah',
            'issue_type': 'technical'
        }
    }
)
dspy.configure(lm=lm)

Learn more about Prompts with AI Gateway.

Advanced Features

Rate Limiting

Configure rate limits for your DSPy applications:

Python

lm = dspy.LM(
    'gpt-4o-mini',
    api_key=os.getenv('HELICONE_API_KEY'),
    api_base='https://ai-gateway.helicone.ai/v1',
    extra_headers={
        'Helicone-Rate-Limit-Policy': 'basic-100'
    }
)

Caching

Enable intelligent caching to reduce costs:

Python

lm = dspy.LM(
    'gpt-4o-mini',
    api_key=os.getenv('HELICONE_API_KEY'),
    api_base='https://ai-gateway.helicone.ai/v1',
    cache=True  # DSPy's built-in caching works with Helicone
)

Session Tracking for Multi-Turn Conversations

Track entire conversation flows in DSPy programs:

Python

import uuid

session_id = str(uuid.uuid4())

lm = dspy.LM(
    'gpt-4o-mini',
    api_key=os.getenv('HELICONE_API_KEY'),
    api_base='https://ai-gateway.helicone.ai/v1',
    extra_headers={
        'Helicone-Session-Id': session_id,
        'Helicone-Session-Name': 'Customer Support',
        'Helicone-Session-Path': '/support/chat'
    }
)
dspy.configure(lm=lm)

# All calls in this session will be grouped together
qa = dspy.ChainOfThought('question -> answer')

# Multiple turns
response1 = qa(question="What is your return policy?")
response2 = qa(question="How long does shipping take?")
response3 = qa(question="Do you ship internationally?")

# View the full conversation in Helicone Sessions

AI Gateway Overview

Learn about Helicone’s AI Gateway features and capabilities

Provider Routing

Configure intelligent routing and automatic failover

Model Registry

Browse all available models and providers

Prompt Management

Version and manage prompts with Helicone Prompts

Custom Properties

Add metadata to track and filter your requests

Sessions

Track multi-turn conversations and user sessions

Rate Limiting

Configure rate limits for your applications

Caching

Reduce costs and latency with intelligent caching

Getting Started

AI Gateway

Observability & Analytics

Prompt Management

Legacy Integrations

References

Introduction

Integration Steps

Complete Working Examples

Basic Chain of Thought

Custom Generation Configuration

Tracking with Custom Properties

Helicone Prompts Integration

Advanced Features

Rate Limiting

Caching

Session Tracking for Multi-Turn Conversations

AI Gateway Overview

Provider Routing

Model Registry

Prompt Management

Custom Properties

Sessions

Rate Limiting

Caching

​Introduction

​Integration Steps

​Complete Working Examples

​Basic Chain of Thought

​Custom Generation Configuration

​Tracking with Custom Properties

​Helicone Prompts Integration

​Advanced Features

​Rate Limiting

​Caching

​Session Tracking for Multi-Turn Conversations

​Related Documentation

AI Gateway Overview

Provider Routing

Model Registry

Prompt Management

Custom Properties

Sessions

Rate Limiting

Caching

Introduction

Integration Steps

Complete Working Examples

Basic Chain of Thought

Custom Generation Configuration

Tracking with Custom Properties

Helicone Prompts Integration

Advanced Features

Rate Limiting

Caching

Session Tracking for Multi-Turn Conversations

Related Documentation