Prompt Management Overview - Helicone OSS LLM Observability

When building LLM applications, you need to manage prompt templates, handle variable substitution, and deploy changes without code deployments. Prompt Management solves this by providing a centralized system for composing, versioning, and deploying prompts with dynamic variables.

Why Prompt Management?

Traditional prompt development involves hardcoded prompts in application code, messy string substitution, and frustrating and rebuilding deployments for every iteration. This creates friction that slows down experimentation and your team’s ability to ship.

Iterate Without Code Changes

Test and deploy prompt changes instantly without rebuilding or redeploying your application

Version Control Built-In

Track every change, compare versions, and rollback instantly if something goes wrong

Dynamic Variables

Use variables anywhere - system prompts, messages, even tool schemas - for truly reusable prompts

Environment Management

Deploy different versions to production, staging, and development environments independently

Quick Start

Create a Prompt

Build a prompt in the Playground. Save any prompt with clear commit histories and tags.

Test and Iterate

Experiment with different variables, inputs, and models until you reach desired output. Variables can be used anywhere, even in tool schemas.

Run Prompt with AI Gateway

Use your prompt instantly by referencing its ID in your AI Gateway. No code changes, no rebuilds.

Prompt Management is available for Chat Completions on the AI Gateway. Simply include prompt_id and inputs in your chat completion requests.

import { OpenAI } from "openai";
import { HeliconeChatCreateParams } from "@helicone/helpers";

const openai = new OpenAI({
  baseURL: "https://ai-gateway.helicone.ai",
  apiKey: process.env.HELICONE_API_KEY,
});

const response = await openai.chat.completions.create({
  model: "gpt-4o-mini",
  prompt_id: "abc123", // Reference your saved prompt
  environment: "production", // Optional: specify environment
  messages: [
    {
      role: "user",
      content: "Hello there!"
    }
  ], // optional: saved prompt also provides messages
  inputs: {
    customer_name: "John Doe",
    product: "AI Gateway"
  }
} as HeliconeChatCreateParams);

import openai
import os

client = openai.OpenAI(
    base_url="https://ai-gateway.helicone.ai",
    api_key=os.environ.get("HELICONE_API_KEY")
)

response = client.chat.completions.create(
    model="gpt-4o-mini",
    prompt_id="abc123",  # Reference your saved prompt
    environment="production",  # Optional: specify environment
    inputs={
        "customer_name": "John Doe",
        "product": "AI Gateway"
    }
)

curl https://ai-gateway.helicone.ai/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $HELICONE_API_KEY" \
  -d '{
    "model": "gpt-4o-mini",
    "prompt_id": "abc123",
    "environment": "production",
    "inputs": {
      "customer_name": "John Doe",
      "product": "AI Gateway"
    }
  }'

Your prompt is automatically compiled with the provided inputs and sent to your chosen model. Update prompts in the dashboard and changes take effect immediately!

Variables

Variables make your prompts dynamic and reusable. Define them once in your prompt template, then provide different values at runtime without changing your code.

Variable Syntax

Variables use the format {{hc:name:type}} where:

name is your variable identifier
type defines the expected data type

{{hc:customer_name:string}}
{{hc:age:number}}
{{hc:is_premium:boolean}}
{{hc:context:any}}

You are a helpful assistant for {{hc:company:string}}.

The customer {{hc:customer_name:string}} is {{hc:age:number}} years old.
Premium status: {{hc:is_premium:boolean}}

Additional context: {{hc:context:any}}

Supported Types

Type	Description	Example Values	Validation
`string`	Text values	`"John Doe"`, `"Hello world"`	None
`number`	Numeric values	`25`, `3.14`, `-10`	AI Gateway type-checking
`boolean`	True/false values	`true`, `false`, `"yes"`, `"no"`	AI Gateway type-checking
`your_type_name`	Any data type	Objects, arrays, strings	None

Only number and boolean types are validated by the Helicone AI Gateway, which will accept strings for any input as long as they can be converted to valid values.

Boolean variables accept multiple formats:

true / false (boolean)
"yes" / "no" (string)
"true" / "false" (string)

Schema Variables

Variables can be used within JSON schemas for tools and response formatting. This enables dynamic schema generation based on runtime inputs.

{
  "name": "moviebot_response",
  "strict": true,
  "schema": {
    "type": "object",
    "properties": {
      "markdown_response": {
        "type": "string"
      },
      "tools_used": {
        "type": "array",
        "items": {
          "type": "string",
          "enum": "{{hc:tools:array}}"
        }
      },
      "user_tier": {
        "type": "string",
        "enum": "{{hc:tiers:array}}"
      }
    },
    "required": [
      "markdown_response",
      "tools_used",
      "user_tier"
    ],
    "additionalProperties": false
  }
}

{
  "tools": ["search", "calculator", "weather"],
  "tiers": ["basic", "premium", "enterprise"]
}

{
  "name": "moviebot_response",
  "strict": true,
  "schema": {
    "type": "object",
    "properties": {
      "markdown_response": {
        "type": "string"
      },
      "tools_used": {
        "type": "array",
        "items": {
          "type": "string",
          "enum": ["search", "calculator", "weather"]
        }
      },
      "user_tier": {
        "type": "string",
        "enum": ["basic", "premium", "enterprise"]
      }
    },
    "required": [
      "markdown_response",
      "tools_used",
      "user_tier"
    ],
    "additionalProperties": false
  }
}

Replacement Behavior

Value Replacement: When a variable tag is the only content in a string, it gets replaced with the actual data type:

"enum": "{{hc:tools:array}}" → "enum": ["search", "calculator", "weather"]

String Substitution: When variables are part of a larger string, normal regex replacement occurs:

"description": "Available for {{hc:name:string}} users" → "description": "Available for premium users"

Keys and Values: Variables work in both JSON keys and values throughout tool schemas and response schemas.

Managing Environments

You can easily manage different deployment environments for your prompts directly in the Helicone dashboard. Create and deploy prompts to production, staging, development, or any custom environment you need.

Prompt Partials

When building multiple prompts, you often need to reuse the same message blocks across different prompts. Prompt partials allow you to reference messages from other prompts, eliminating duplication and making your prompt library more maintainable.

Syntax

Prompt partials use the format {{hcp:prompt_id:index:environment}} where:

prompt_id - The 6-character alphanumeric identifier of the prompt to reference
index - The message index (0-based) to extract from that prompt
environment - Optional environment identifier (defaults to production if omitted)

{{hcp:abc123:0}}                   // Message 0 from prompt abc123 (production)
{{hcp:abc123:1:staging}}           // Message 1 from prompt abc123 (staging)
{{hcp:xyz789:2:development}}       // Message 2 from prompt xyz789 (development)

{{hcp:abc123:0}}

{{hc:user_name:string}}, here's your personalized response:

How It Works

When a prompt containing a partial is compiled:

Partial Resolution: The partial tag {{hcp:prompt_id:index:environment}} is replaced with the actual message content from the referenced prompt at the specified index
Variable Substitution: After partials are resolved, variables in both the main prompt and the resolved partials are substituted with their values

This order matters: since partials are resolved before variables, you can control variables that exist within the partial from the main prompt’s inputs.

{
  "messages": [
    {
      "role": "system",
      "content": "You are a helpful assistant for {{hc:company:string}}."
    }
  ]
}

{
  "messages": [
    {
      "role": "user",
      "content": "{{hcp:abc123:0}} Please help me with my account."
    }
  ]
}

{
  "company": "Acme Corp"
}

{
  "role": "user",
  "content": "You are a helpful assistant for Acme Corp. Please help me with my account."
}

Variables from partials are automatically extracted and shown in the prompt editor. You can provide values for these variables just like any other prompt variable, giving you full control over the partial’s content.

Using Prompts

Helicone provides two ways to use prompts:

AI Gateway Integration - The recommended approach. Use prompts through the Helicone AI Gateway for automatic compilation, input tracing, and lower latency.
SDK Integration - Alternative integration method for users that need direct interaction with compiled prompt bodies without using the AI Gateway.

Prompt Management is available for Chat Completions on the AI Gateway. Simply include prompt_id and inputs in your chat completion requests to use saved prompts.

Learn more about how prompts are assembled and compiled in the Prompt Assembly guide.

Prompt Assembly

Understand how prompts are compiled from templates and runtime parameters

SDK Integration

Use prompts directly via SDK without the AI Gateway

AI Gateway

Learn about prompt integration with the AI Gateway

Playground

Create and test prompts in the Helicone dashboard

​Why Prompt Management?

Iterate Without Code Changes

Version Control Built-In

Dynamic Variables

Environment Management

​Quick Start

​Variables

​Variable Syntax

​Supported Types

​Schema Variables

​Replacement Behavior

​Managing Environments

​Prompt Partials

​Syntax

​How It Works

​Using Prompts

​Related Documentation

Prompt Assembly

SDK Integration

AI Gateway

Playground

Why Prompt Management?

Quick Start

Variables

Variable Syntax

Supported Types

Schema Variables

Replacement Behavior

Managing Environments

Prompt Partials

Syntax

How It Works

Using Prompts

Related Documentation