User Feedback

When building AI applications, you need real-world signals about response quality to improve prompts, catch regressions, and understand what users find helpful. User Feedback lets you collect positive/negative ratings on LLM responses, enabling data-driven improvements to your AI systems based on actual user satisfaction.

Why use User Feedback

Improve response quality: Identify patterns in poorly-rated responses to refine prompts and model selection
Catch regressions early: Monitor feedback trends to detect when changes negatively impact user experience
Build training datasets: Use highly-rated responses as examples for fine-tuning or few-shot prompting

Quick Start

Make a request and capture the ID

Capture the Helicone request ID from your LLM response:

import OpenAI from "openai";

const openai = new OpenAI({
  apiKey: process.env.OPENAI_API_KEY,
  baseURL: "https://oai.helicone.ai/v1",
  defaultHeaders: {
    "Helicone-Auth": `Bearer ${process.env.HELICONE_API_KEY}`,
  },
});

// Use a custom request ID for feedback tracking
const customId = crypto.randomUUID();

const response = await openai.chat.completions.create({
  model: "gpt-4o-mini",
  messages: [{ role: "user", content: "Explain quantum computing" }]
}, {
  headers: {
    "Helicone-Request-Id": customId
  }
});

// Use your custom ID for feedback
const heliconeId = customId;

Alternative: Getting request ID from response

You can also try to get the Helicone ID from response headers, though this may not always be available:

const response = await openai.chat.completions.create({
  model: "gpt-4o-mini",
  messages: [{ role: "user", content: "Explain quantum computing" }]
});

// Try to get the Helicone request ID from response headers
const heliconeId = response.response?.headers?.get("helicone-id");

// If not available, you'll need to use a custom ID approach
if (!heliconeId) {
  console.log("Helicone ID not found in headers, use custom ID approach instead");
}

Submit feedback rating

Send a positive or negative rating for the response:

const feedback = await fetch(
  `https://api.helicone.ai/v1/request/${heliconeId}/feedback`,
  {
    method: "POST",
    headers: {
      "Authorization": `Bearer ${process.env.HELICONE_API_KEY}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({
      rating: true  // true = positive, false = negative
    }),
  }
);

View feedback analytics

Access feedback metrics in your Helicone dashboard to analyze response quality trends and identify areas for improvement.

Configuration Options

Feedback collection requires minimal configuration:

Parameter	Type	Description	Default	Example
`rating`	`boolean`	User’s feedback on the response	N/A	`true` (positive) or `false` (negative)
`helicone-id`	`string`	Request ID to attach feedback to	N/A	UUID

Processing multiple feedback ratings

When you need to submit feedback for multiple requests, use parallel API calls:

// Note: There is no bulk feedback endpoint - each rating requires a separate API call
const feedbackBatch = [
  { requestId: "f47ac10b-58cc-4372-a567-0e02b2c3d479", rating: true },
  { requestId: "6ba7b810-9dad-11d1-80b4-00c04fd430c8", rating: false },
  { requestId: "6ba7b811-9dad-11d1-80b4-00c04fd430c8", rating: true }
];

// Submit feedback in parallel for better performance
const feedbackPromises = feedbackBatch.map(({ requestId, rating }) =>
  fetch(`https://api.helicone.ai/v1/request/${requestId}/feedback`, {
    method: "POST",
    headers: {
      "Authorization": `Bearer ${process.env.HELICONE_API_KEY}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({ rating }),
  })
);

// Wait for all feedback submissions to complete
const results = await Promise.all(feedbackPromises);

// Check for any failed submissions
results.forEach((result, index) => {
  if (!result.ok) {
    console.error(`Failed to submit feedback for request ${feedbackBatch[index].requestId}`);
  }
});

Use Cases

Chat Application Quality
Code Generation Evaluation
Customer Support Bot

Track user satisfaction with AI assistant responses:

import OpenAI from "openai";

const openai = new OpenAI({
  apiKey: process.env.OPENAI_API_KEY,
  baseURL: "https://oai.helicone.ai/v1",
  defaultHeaders: {
    "Helicone-Auth": `Bearer ${process.env.HELICONE_API_KEY}`,
  },
});

// In your chat handler
async function handleChatMessage(userId: string, message: string) {
  const requestId = crypto.randomUUID();
  
  const response = await openai.chat.completions.create(
    {
      model: "gpt-4o-mini",
      messages: [
        { role: "system", content: "You are a helpful assistant." },
        { role: "user", content: message }
      ]
    },
    {
      headers: {
        "Helicone-Request-Id": requestId,
        "Helicone-User-Id": userId,
        "Helicone-Property-Feature": "chat"
      }
    }
  );

  // Store request ID for later feedback
  await storeRequestMapping(userId, requestId, response.id);
  
  return response;
}

// When user clicks thumbs up/down
async function handleUserFeedback(userId: string, responseId: string, isPositive: boolean) {
  const requestId = await getRequestId(userId, responseId);
  
  await fetch(
    `https://api.helicone.ai/v1/request/${requestId}/feedback`,
    {
      method: "POST",
      headers: {
        "Authorization": `Bearer ${process.env.HELICONE_API_KEY}`,
        "Content-Type": "application/json",
      },
      body: JSON.stringify({ rating: isPositive }),
    }
  );
}

import openai
import uuid
import requests

client = openai.OpenAI(
    api_key=os.environ.get("OPENAI_API_KEY"),
    base_url="https://oai.helicone.ai/v1",
    default_headers={
        "Helicone-Auth": f"Bearer {os.environ.get('HELICONE_API_KEY')}",
    }
)

def handle_chat_message(user_id: str, message: str):
    request_id = str(uuid.uuid4())
    
    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[
            {"role": "system", "content": "You are a helpful assistant."},
            {"role": "user", "content": message}
        ],
        extra_headers={
            "Helicone-Request-Id": request_id,
            "Helicone-User-Id": user_id,
            "Helicone-Property-Feature": "chat"
        }
    )
    
    # Store mapping for later feedback
    store_request_mapping(user_id, request_id, response.id)
    return response

def handle_user_feedback(user_id: str, response_id: str, is_positive: bool):
    request_id = get_request_id(user_id, response_id)
    
    response = requests.post(
        f"https://api.helicone.ai/v1/request/{request_id}/feedback",
        headers={
            "Authorization": f"Bearer {os.environ.get('HELICONE_API_KEY')}",
            "Content-Type": "application/json",
        },
        json={"rating": is_positive}
    )

Collect feedback on generated code quality:

// After generating code for the user
const codeGenResponse = await openai.chat.completions.create(
  {
    model: "gpt-4o-mini",
    messages: [
      { role: "system", content: "You are an expert programmer." },
      { role: "user", content: `Generate a ${language} function to ${task}` }
    ]
  },
  {
    headers: {
      "Helicone-Auth": `Bearer ${process.env.HELICONE_API_KEY}`,
      "Helicone-Property-Feature": "code-generation",
      "Helicone-Property-Language": language
    }
  }
);

// Track if the generated code worked
const codeWorked = await userTestedCode(); // Your logic here

// Auto-submit feedback based on code execution
const heliconeId = codeGenResponse.headers?.get("helicone-id");
if (heliconeId) {
  await fetch(
    `https://api.helicone.ai/v1/request/${heliconeId}/feedback`,
    {
      method: "POST",
      headers: {
        "Authorization": `Bearer ${process.env.HELICONE_API_KEY}`,
        "Content-Type": "application/json",
      },
      body: JSON.stringify({ rating: codeWorked }),
    }
  );
}

// Analyze which languages/tasks have highest success rates

Measure effectiveness of automated support responses:

// Support ticket handler
async function handleSupportQuery(ticketId: string, query: string) {
  const requestId = `ticket-${ticketId}-${Date.now()}`;
  
  const response = await openai.chat.completions.create(
    {
      model: "gpt-4o-mini",
      messages: [
        { 
          role: "system", 
          content: "You are a technical support specialist. Provide clear, helpful solutions." 
        },
        { role: "user", content: query }
      ],
      temperature: 0.3 // Lower temperature for consistent support answers
    },
    {
      headers: {
        "Helicone-Auth": `Bearer ${process.env.HELICONE_API_KEY}`,
        "Helicone-Request-Id": requestId,
        "Helicone-Property-Type": "support",
        "Helicone-Property-TicketId": ticketId
      }
    }
  );

  // Send response to user
  await sendSupportResponse(ticketId, response.choices[0].message.content);
  
  // Follow up after resolution
  setTimeout(async () => {
    const wasHelpful = await checkIfTicketResolved(ticketId);
    
    await fetch(
      `https://api.helicone.ai/v1/request/${requestId}/feedback`,
      {
        method: "POST",
        headers: {
          "Authorization": `Bearer ${process.env.HELICONE_API_KEY}`,
          "Content-Type": "application/json",
        },
        body: JSON.stringify({ rating: wasHelpful }),
      }
    );
  }, 24 * 60 * 60 * 1000); // Check after 24 hours
}

Understanding User Feedback

How it works

User feedback creates a continuous improvement loop for your AI application:

Each LLM request gets a unique Helicone ID
Users rate responses as positive (helpful) or negative (not helpful)
Feedback is linked to the original request for analysis
Dashboard aggregates feedback to show quality trends

Explicit vs Implicit Feedback

Explicit feedback is when users directly rate responses (thumbs up/down, star ratings). While valuable, it has low response rates since users must take deliberate action. Implicit feedback is derived from user behavior and is much more valuable since it reflects actual usage patterns: Track user actions that indicate response quality:

// Code completion acceptance (like Cursor)
async function trackCodeCompletion(requestId: string, suggestion: string) {
  // Monitor if user accepts the completion
  const accepted = await waitForUserAction(suggestion);
  
  await fetch(`https://api.helicone.ai/v1/request/${requestId}/feedback`, {
    method: "POST",
    headers: {
      "Authorization": `Bearer ${process.env.HELICONE_API_KEY}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({ 
      rating: accepted // true if accepted, false if rejected/ignored
    }),
  });
}

// Chat engagement patterns
async function trackChatEngagement(requestId: string, response: string) {
  // Track user behavior after response
  const userActions = await monitorUserBehavior(60000); // 1 minute
  
  const implicitRating = 
    userActions.continuedConversation || // User asked follow-up
    userActions.copiedResponse ||        // User copied the answer
    userActions.sharedResponse ||        // User shared/saved
    userActions.timeSpent > 30;          // User read for >30 seconds
  
  await submitFeedback(requestId, implicitRating);
}

// Search/recommendation clicks
async function trackSearchResult(requestId: string, results: string[]) {
  // Monitor if user clicks on suggested results
  const clicked = await trackClicks(results, 300000); // 5 minutes
  
  // High click-through rate = good recommendations
  const rating = clicked.length > 0;
  await submitFeedback(requestId, rating);
}

Custom Properties

Segment feedback by feature, user type, or experiment for deeper insights

User Metrics

Combine feedback with usage data to understand user satisfaction trends

Sessions

Track feedback across multi-turn conversations and workflows

Alerts

Set up notifications when feedback rates drop below thresholds

Getting Started

AI Gateway

Observability & Analytics

Prompt Management

Legacy Integrations

References

Why use User Feedback

Quick Start

Configuration Options

Use Cases

Understanding User Feedback

How it works

Explicit vs Implicit Feedback

Custom Properties

User Metrics

Sessions

Alerts

​Why use User Feedback

​Quick Start

​Configuration Options

​Use Cases

​Understanding User Feedback

​How it works

​Explicit vs Implicit Feedback

​Related Features

Custom Properties

User Metrics

Sessions

Alerts

Why use User Feedback

Quick Start

Configuration Options

Use Cases

Understanding User Feedback

How it works

Explicit vs Implicit Feedback

Related Features