​
Background

There are two different ways to interface with Helicone. Proxy and Async. Both have their own advantages and disadvantages. This page will help you determine which one is right for you.

ProxyAsync
Easy setup✅❌
Custom rate limiting✅❌
Bucket Cache✅❌
Retries✅❌
Not on critical path❌✅
0 Propagation Delay❌✅
Open source✅✅
Custom Properties✅✅
User Metrics✅✅
Negligible Logging Delay✅✅
Streaming Support✅✅
Prompts✅✅
Prompts Auto Formatting (easier)✅❌

​
Proxy

Helicone Proxy allows it’s user to easily integrate, as users only have to replace their call to the LLM with a call to the proxy. The proxy will then forward the request to the LLM and return the response to the user. Since the proxy sits on the edge and is the gate keeper of the requests we are able to instrument a suite of tools like Caching, Backoff and Rate limits directly into the proxy.

​
Async

Helicone Async allows for a more flexible workflow where the actual logging of the event is not on the critical path. This gives some users more confidence that if we are going down or if there is a network issue that it will not affect their application. The downside is that we are not able to instrument the same suite of tools as we can with the proxy.

Quick StartSelf Deploy