TypeScript Manual Logger

Logging calls to custom models is supported via the Helicone NodeJS SDK.

Basic Usage
Logging Chat Completion
On Vercel

To get started, install the `@helicone/helpers` package

npm install @helicone/helpers

Set `HELICONE_API_KEY` as an environment variable

export HELICONE_API_KEY=sk-<your-api-key>

You can also set the Helicone API Key in your code (See below)

Create a new HeliconeManualLogger instance

import { HeliconeManualLogger } from "@helicone/helpers";

const heliconeLogger = new HeliconeManualLogger({
apiKey: process.env.HELICONE_API_KEY, // Can be set as env variable
headers: {} // Additional headers to be sent with the request
});

Log your request

const reqBody = {
  model: "text-embedding-ada-002",
  input: "The food was delicious and the waiter was very friendly.",
  encoding_format: "float"
}
const res = await heliconeLogger.logRequest(
  reqBody,
  async (resultRecorder) => {
    const r = await fetch("https://api.openai.com/v1/embeddings", {
      method: "POST",
      headers: {
        "Content-Type": "application/json",
        Authorization: `Bearer ${process.env.OPENAI_API_KEY}`
      },
      body: JSON.stringify(reqBody)
    })
    const resBody = await r.json();
    resultRecorder.appendResults(resBody);
    return resBody; // this will be returned by the logRequest function
  },
  {
   // Additional headers to be sent with the request
  }
);

Install dependencies

npm install @helicone/helpers openai

Set up the logger and OpenAI client

import { HeliconeManualLogger } from "@helicone/helpers";
import OpenAI from "openai";

// Initialize the Helicone logger
const helicone = new HeliconeManualLogger({
  apiKey: process.env.HELICONE_API_KEY!,
});

// Initialize the OpenAI client
const openai = new OpenAI({
  apiKey: process.env.OPENAI_API_KEY!,
});

Using logSingleRequest for non-streaming responses

// Define your request
const requestBody = {
  model: "gpt-4o-mini",
  messages: [
    { role: "user", content: "Explain quantum computing in simple terms" },
  ],
};

// Make the API call
const response = await openai.chat.completions.create(requestBody);

// Log the request and response to Helicone
await helicone.logSingleRequest(requestBody, JSON.stringify(response), {
  additionalHeaders: { "Helicone-User-Id": "user-123" }, // Optional additional headers
});

console.log(response.choices[0].message.content);

Using logSingleStream for streaming responses

const streamingRequestBody = {
  model: "gpt-4o-mini",
  messages: [{ role: "user", content: "Write a short story about AI" }],
  stream: true,
};

const streamingResponse = await openai.chat.completions.create(
  streamingRequestBody
);

const [streamForUser, streamForLogging] = stream.tee();

helicone.logSingleStream(streamingRequestBody, streamForLogging, {
  "Helicone-User-Id": "user-123",
});

Install dependencies

npm install @helicone/helpers together-ai next

Create an API route handler with Together AI

// app/api/chat/route.ts
import { HeliconeManualLogger } from "@helicone/helpers";
import { after } from "next/server";
import Together from "together-ai";

export async function POST(request: Request) {
const { question } = await request.json();

  const together = new Together();
  const helicone = new HeliconeManualLogger({
    apiKey: process.env.HELICONE_API_KEY!,

  });

  const nonStreamingBody = {
    model: "meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo",
    messages: [{ role: "user", content: question }],
    stream: false,
  } as Together.Chat.CompletionCreateParamsNonStreaming & { stream: false };

  const completion = await together.chat.completions.create(nonStreamingBody);

  after(
    helicone.logSingleRequest(nonStreamingBody, JSON.stringify(completion), {
      additionalHeaders: { "Helicone-User-Id": "123" },
    }),
  );

  const body = {
    model: "meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo",
    messages: [{ role: "user", content: question }],
    stream: true,
  } as Together.Chat.CompletionCreateParamsStreaming & { stream: true };

  const response = await together.chat.completions.create(body);
  const [stream1, stream2] = response.tee();
  after(
    helicone.logSingleStream(body, stream2.toReadableStream(), {
      "Helicone-User-Id": "123",
    }),
  );

  return new Response(stream1.toReadableStream());

}

Important note about the `after` function

The after function allows you to perform operations after the response has been sent to the client. This is crucial for logging operations as it ensures they don’t delay the response to the user.When using this approach:

Logging happens asynchronously after the response is sent
The user experience isn’t affected by logging latency
You still capture all the necessary data for observability

This is especially important for streaming responses where any delay would be noticeable to the user.

API Reference

HeliconeManualLogger

class HeliconeManualLogger {
constructor(opts: IHeliconeManualLogger);
}

type IHeliconeManualLogger = {
apiKey: string;
headers?: Record<string, string>;
loggingEndpoint?: string; // defaults to https://api.hconeai.com/custom/v1/log
};

HeliconeLogBuilder

class HeliconeLogBuilder {
  constructor(
    logger: HeliconeManualLogger,
    request: HeliconeLogRequest,
    additionalHeaders?: Record<string, string>
  );

  setError(error: any): void;
  toReadableStream<T>(stream: Stream<T>): ReadableStream;
  setResponse(body: string): void;
  sendLog(): Promise<void>;
}

The HeliconeLogBuilder provides a simplified way to handle streaming LLM responses with better error handling and async support. It’s created using the logBuilder method of HeliconeManualLogger.

Methods

setError(error: any): Sets an error that occurred during the request
toReadableStream<T>(stream: Stream<T>): Collects streaming responses and converts them to a readable stream while capturing the response for logging
setResponse(body: string): Sets the response body for non-streaming responses
sendLog(): Sends the log to Helicone

logRequest

logRequest<T>(
    request: HeliconeLogRequest,
    operation: (resultRecorder: HeliconeResultRecorder) => Promise<T>,
    additionalHeaders?: Record<string, string>
  ): Promise<T>

Parameters

request: HeliconeLogRequest - The request object to log

type HeliconeLogRequest = ILogRequest | HeliconeCustomEventRequest; // ILogRequest is the type for the request object for custom model logging

// The name and structure of the prompt field depends on the model you are using.
// Eg: for chat models it is named "messages", for embeddings models it is named "input".
// Hence, the only enforced type is `model`, you need still add the respective prompt property for your model.
// You may also add more properties (eg: temperature, stop reason etc)
type ILogRequest = {
  model: string;
  [key: string]: any;
};

operation: (resultRecorder: HeliconeResultRecorder) => Promise<T> - The operation to be executed and logged

class HeliconeResultRecorder {
  private results: Record<string, any> = {};

  appendResults(data: Record<string, any>): void {
    this.results = { ...this.results, ...data };
  }

  getResults(): Record<string, any> {
    return this.results;
  }
}

additionalHeaders: Record<string, string>
- Additional headers to be sent with the request
- This can be used to use features like session management, custom properties, etc.

Available Methods

The HeliconeManualLogger class provides several methods for logging different types of requests and responses. Here’s a comprehensive overview of each method:

logRequest

Used for logging non-streaming requests and responses with full control over the operation.

logRequest<T>(
  request: HeliconeLogRequest,
  operation: (resultRecorder: HeliconeResultRecorder) => Promise<T>,
  additionalHeaders?: Record<string, string>
): Promise<T>

Parameters:

request: The request object to log
operation: A function that performs the actual API call and records the results
additionalHeaders: Optional additional headers to include with the log request

Example:

const result = await helicone.logRequest(
  requestBody,
  async (resultRecorder) => {
    const response = await llmProvider.createCompletion(requestBody);
    resultRecorder.appendResults(response);
    return response;
  },
  { "Helicone-User-Id": userId }
);

logStream

Used for logging streaming operations with full control over stream handling.

logStream<T>(
  request: HeliconeLogRequest,
  operation: (resultRecorder: HeliconeStreamResultRecorder) => Promise<T>,
  additionalHeaders?: Record<string, string>
): Promise<T>

Parameters:

request: The request object to log
operation: A function that performs the streaming API call and attaches the stream to the recorder
additionalHeaders: Optional additional headers to include with the log request

Example:

const stream = await helicone.logStream(
  requestBody,
  async (resultRecorder) => {
    const response = await llmProvider.createChatCompletion({
      stream: true,
      ...requestBody,
    });
    const [stream1, stream2] = response.tee();
    resultRecorder.attachStream(stream2.toReadableStream());
    return stream1;
  },
  { "Helicone-User-Id": userId }
);

logSingleStream

A simplified method for logging a single ReadableStream without needing to manage the operation.

logSingleStream(
  request: HeliconeLogRequest,
  stream: ReadableStream,
  additionalHeaders?: Record<string, string>
): Promise<void>

Parameters:

request: The request object to log
stream: The ReadableStream to consume and log
additionalHeaders: Optional additional headers to include with the log request

Example:

const response = await llmProvider.createChatCompletion({
  stream: true,
  ...requestBody,
});
const stream = response.toReadableStream();
const [streamForUser, streamForLogging] = stream.tee();

helicone.logSingleStream(requestBody, streamForLogging, {
  "Helicone-User-Id": userId,
});

return streamForUser;

logSingleRequest

Used for logging a single request with a response body without needing to manage the operation.

logSingleRequest(
  request: HeliconeLogRequest,
  body: string,
  options: {
    additionalHeaders?: Record<string, string>;
    latencyMs?: number;
  }
): Promise<void>

Parameters:

request: The request object to log
body: The response body as a string
additionalHeaders: Optional additional headers to include with the log request

Example:

const response = await llmProvider.createCompletion(requestBody);
await helicone.logSingleRequest(requestBody, JSON.stringify(response), {
  additionalHeaders: { "Helicone-User-Id": userId },
});

logBuilder

The recommended method for handling streaming responses with better error handling and simplified workflow.

logBuilder(
  request: HeliconeLogRequest,
  additionalHeaders?: Record<string, string>
): HeliconeLogBuilder

Parameters:

request: The request object to log
additionalHeaders: Optional additional headers to include with the log request

Example:

// Create a log builder
const heliconeLogBuilder = helicone.logBuilder(requestBody, {
  "Helicone-User-Id": userId,
});

try {
  // Make the LLM API call
  const response = await llmProvider.createChatCompletion({
    stream: true,
    ...requestBody,
  });

  // Convert the API response to a readable stream and return it
  return new Response(heliconeLogBuilder.toReadableStream(response));
} catch (error) {
  // Record any errors that occur
  heliconeLogBuilder.setError(error);
  throw error;
} finally {
  // Send the log (can be used with Vercel's "after" function)
  await heliconeLogBuilder.sendLog();
}

Streaming Examples

Using the Async Stream Parser

Helicone provides an asynchronous stream parser for efficient handling of streamed responses. This is particularly useful when working with custom integrations that support streaming. Here’s an example of how to use the async stream parser with a custom integration:

import { HeliconeManualLogger } from "@helicone/helpers";

// Initialize the Helicone logger
const heliconeLogger = new HeliconeManualLogger({
  apiKey: process.env.HELICONE_API_KEY!,
  headers: {}, // You can add custom headers here
});

// Your custom model API call that returns a stream
const response = await customModelAPI.generateStream(prompt);

// If your API supports splitting the stream
const [stream1, stream2] = response.tee();

// Log the stream to Helicone using the async stream parser
heliconeLogger.logStream(requestBody, async (resultRecorder) => {
  resultRecorder.attachStream(stream1);
});

// Process the stream for your application
for await (const chunk of stream2) {
  console.log(chunk);
}

The async stream parser offers several benefits:

Processes stream chunks asynchronously for better performance
Reduces latency when handling large streamed responses
Provides more reliable token counting for streamed content

Using Vercel’s `after` Function with Streaming

When building applications with Next.js App Router on Vercel, you can use the after function to log streaming responses without blocking the client response:

import { HeliconeManualLogger } from "@helicone/helpers";
import { after } from "next/server";
import Together from "together-ai";

export async function POST(request: Request) {
  const { prompt } = await request.json();

  const together = new Together({ apiKey: process.env.TOGETHER_API_KEY });
  const helicone = new HeliconeManualLogger({
    apiKey: process.env.HELICONE_API_KEY!,
  });

  // Example with non-streaming response
  const nonStreamingBody = {
    model: "meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo",
    messages: [{ role: "user", content: prompt }],
    stream: false,
  };

  const completion = await together.chat.completions.create(nonStreamingBody);

  // Log non-streaming response after sending the response to the client
  after(
    helicone.logSingleRequest(nonStreamingBody, JSON.stringify(completion))
  );

  // Example with streaming response
  const streamingBody = {
    model: "meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo",
    messages: [{ role: "user", content: prompt }],
    stream: true,
  };

  const response = await together.chat.completions.create(streamingBody);
  const [stream1, stream2] = response.tee();

  // Log streaming response after sending the response to the client
  after(helicone.logSingleStream(streamingBody, stream2.toReadableStream()));

  return new Response(stream1.toReadableStream());
}

For a comprehensive guide on using the Manual Logger with streaming functionality, check out our Manual Logger with Streaming cookbook.

Getting Started

AI Gateway

Observability & Analytics

Prompt Management

Legacy Integrations

References

Manual Logger - TypeScript

TypeScript Manual Logger

API Reference

HeliconeManualLogger

HeliconeLogBuilder

Methods

logRequest

Parameters

Available Methods

logRequest

logStream

logSingleStream

logSingleRequest

logBuilder

Streaming Examples

Using the Async Stream Parser

Using Vercel’s `after` Function with Streaming

​TypeScript Manual Logger

​API Reference

​HeliconeManualLogger

​HeliconeLogBuilder

​Methods

​logRequest

​Parameters

​Available Methods

​logRequest

​logStream

​logSingleStream

​logSingleRequest

​logBuilder

​Streaming Examples

​Using the Async Stream Parser

​Using Vercel’s after Function with Streaming

TypeScript Manual Logger

API Reference

HeliconeManualLogger

HeliconeLogBuilder

Methods

logRequest

Parameters

Available Methods

logRequest

logStream

logSingleStream

logSingleRequest

logBuilder

Streaming Examples

Using the Async Stream Parser

Using Vercel’s `after` Function with Streaming