Prompt Management

​

Why Prompt Management?

• Powerful composability: Variables of all types in system prompts, messages, and tool/response schemas
• Clear version control: Track, compare, and rollback prompt versions without code changes
• Environment management: Full control over deployment environments like production, staging, and development
• Easy deployment: Reference prompts by ID and let our AI Gateway handle the rest
• Real-time testing: Test prompts instantly with different models and parameters

​

Quickstart

1

Create a Prompt

Build a prompt in the Playground. Save any prompt with clear commit histories and tags.

2

Test and Iterate

Experiment with different variables, inputs, and models until you reach desired output. Variables can be used anywhere, even in tool schemas.

3

Run Prompt with AI Gateway

Use your prompt instantly by referencing its ID in your AI Gateway. No code changes, no rebuilds.

TypeScript

Python

cURL

Copy

Ask AI

import { OpenAI } from "openai";

const openai = new OpenAI({
  baseURL: "https://ai-gateway.helicone.ai/v1",
  apiKey: "your-openai-api-key",
  defaultHeaders: {
    "Helicone-Auth": "Bearer your-helicone-api-key",
  },
});

const response = await openai.chat.completions.create({
  model: "openai/gpt-4o-mini",
  prompt_id: "abc123", // Reference your saved prompt
  environment: "production", // Optional: specify environment
  inputs: {
    customer_name: "John Doe",
    product: "AI Gateway"
  }
});

​

Prompt Assembly Process

​

Version Selection

1. Environment specified: If you provide an environment parameter, the gateway uses the prompt version deployed to that environment (e.g., “production”, “staging”, “development”)
2. Version ID specified: If you provide a version_id parameter but no environment, the gateway uses that specific version
3. Default behavior: If neither environment nor version ID is specified, the gateway automatically uses the production version

​

Parameter Priority

{
  "model": "gpt-4o-mini",
  "temperature": 0.6,
  "max_tokens": 1000,
  "messages": [
    {
      "role": "system", 
      "content": "You are a helpful customer support agent for {{hc:company:string}}."
    },
    {
      "role": "user",
      "content": "Hello, I need help with my account."
    }
  ]
}

​

Message Handling

• Define consistent system prompts and example conversations in your saved prompt
• Add dynamic user messages at runtime
• Build multi-turn conversations that maintain context

​

Override Examples

// Saved prompt has temperature: 0.8
const response = await openai.chat.completions.create({
  prompt_id: "abc123",
  temperature: 0.2, // Uses 0.2, not 0.8
  inputs: { topic: "AI safety" }
});

​

Managing Environments

​

Variables

​

Variable Syntax

• name is your variable identifier
• type defines the expected data type

{{hc:customer_name:string}}
{{hc:age:number}}
{{hc:is_premium:boolean}}
{{hc:context:any}}

​

Supported Types

• true / false (boolean)
• "yes" / "no" (string)
• "true" / "false" (string)

​

Schema Variables

{
  "name": "moviebot_response",
  "strict": true,
  "schema": {
    "type": "object",
    "properties": {
      "markdown_response": {
        "type": "string"
      },
      "tools_used": {
        "type": "array",
        "items": {
          "type": "string",
          "enum": "{{hc:tools:array}}"
        }
      },
      "user_tier": {
        "type": "string",
        "enum": "{{hc:tiers:array}}"
      }
    },
    "required": [
      "markdown_response",
      "tools_used",
      "user_tier"
    ],
    "additionalProperties": false
  }
}

​

Replacement Behavior

"enum": "{{hc:tools:array}}" → "enum": ["search", "calculator", "weather"]

"description": "Available for {{hc:name:string}} users" → "description": "Available for premium users"

​

SDK Helpers

1. AI Gateway Integration - Use prompts through the Helicone AI Gateway
2. Direct SDK Integration - Pull prompts directly via SDK

• Cleaner code: Automatically performs compilation and substitution in the router.
• Input traces: Traces inputs on each request for better observability in Helicone requests.
• Faster TTFT: The AI Gateway adds significantly less latency compared to the SDK.

npm install @helicone/helpers

​

Types and Classes

import { HeliconePromptManager } from '@helicone/helpers';

const promptManager = new HeliconePromptManager({
  apiKey: "your-helicone-api-key"
});

​

Methods

​

Usage Examples

import OpenAI from 'openai';
import { HeliconePromptManager } from '@helicone/helpers';

const openai = new OpenAI({
  apiKey: "your-openai-api-key",
  baseURL: "https://oai.helicone.ai/v1",
  defaultHeaders: {
    "Helicone-Auth": "Bearer your-helicone-api-key",
  },
});

const promptManager = new HeliconePromptManager({
  apiKey: "your-helicone-api-key"
});

async function generateWithPrompt() {
  // Get compiled prompt with variable substitution
  const { body, errors } = await promptManager.getPromptBody({
    prompt_id: "abc123",
    model: "gpt-4o-mini",
    inputs: {
      customer_name: "Alice Johnson",
      product: "AI Gateway"
    }
  });

  // Check for validation errors
  if (errors.length > 0) {
    console.warn("Validation errors:", errors);
  }

  // Use compiled prompt with OpenAI SDK
  const response = await openai.chat.completions.create(body);
  console.log(response.choices[0].message.content);
}