​
Introduction

Retrying requests is a common best practice when dealing with overloaded servers or hitting rate limits. These issues typically manifest as HTTP status codes 429 (Too Many Requests) and 500 (Internal Server Error). For more information on error codes, see the OpenAI API error codes documentation.

​
Why Retries

  • Overcoming rate limits and server overload.
  • Reducing the load on the server, increasing the likelihood of request success on subsequent attempts.

​
Quick Start

To get started, set Helicone-Retry-Enabled to true. 
curl --request POST \
     --url https://oai.helicone.ai/v1/chat/completions \
     -H "Content-Type: application/json" \
     -H "Authorization: Bearer $OPENAI_API_KEY" \
     -H "Helicone-Auth: Bearer $HELICONE_API_KEY" \
     -H "Helicone-Retry-Enabled: true" \
     --data '{
        "model": "gpt-4o-mini",
        "messages": [
          {
            "role": "system",
            "content": "Say Hello!"
            }
        ],
        "temperature": 1,
        "max_tokens": 30
      }'
When a retry happens, the request will be logged in Helicone.

​
Retries Parameters

You can customize the behavior of the retries feature by setting additional headers in your request.
ParameterDescriptionDefault Value
helicone-retry-numNumber of retries5
helicone-retry-factorThe exponential backoff factor used to increaase the wait time between subsequent retries. The default is usually 2.2
helicone-retry-min-timeoutMinimum timeout (in milliseconds) between retries1000
helicone-retry-max-timeoutMaximum timeout (in milliseconds) between retries10000
Header values have to be strings. For example, "helicone-retry-num": "3".