Kubernetes Self-Hosting
Deploy Helicone using Kubernetes and Helm. Quick setup guide for running a containerized instance of the LLM observability platform on your Kubernetes cluster.
The Helm chart deploys the complete Helicone stack on Kubernetes. Terraform creates AWS S3, Aurora, and EKS resources to run the Helicone project on. The Helm chart is available in the Helicone Helm repository. Previous version: v2
AWS Setup Guide
Prerequisites
- Install AWS CLI - Install and configure with appropriate permissions
- Install kubectl - For Kubernetes operations
- Install Helm - For chart deployment
- Install Terraform - For infrastructure as code deployment
- Copy all values.example.yaml files to values.yaml for each of the charts in
charts/
and customize as needed for your configuration.
Cluster Creation on EKS with Terraform
- Set up Terraform
- Go to terraform/eks, then
terraform init
, followed byterrform validate
followed byterraform apply
Deploy Helm Charts
Option 1: Using Helm Compose (Recommended)
You can now deploy all Helicone components with a single command using the provided
helm-compose.yaml
configuration:
This will deploy the complete Helicone stack including:
- helicone-core - Main application components (web, jawn, worker, etc.)
- helicone-infrastructure - Infrastructure services (PostgreSQL, Redis, ClickHouse, etc.)
- helicone-monitoring - Monitoring stack (Grafana, Prometheus)
- helicone-argocd - ArgoCD for GitOps workflows
To tear down all components:
Option 2: Manual Helm Installation
Alternatively, you can install components individually:
-
Install necessary helm dependencies:
-
Use
values.example.yaml
as a starting point, and copy intovalues.yaml
-
Copy
secrets.example.yaml
intosecrets.yaml
, and change the secrets according to your setup. -
Install/upgrade each Helm chart individually:
-
Verify the deployment:
Accessing Deployed Services
ArgoCD
ArgoCD is deployed as part of the helicone-argocd component and provides GitOps capabilities for continuous deployment. It monitors your Git repositories and automatically synchronizes your Kubernetes cluster state with the desired state defined in your Git repos.
Accessing ArgoCD UI
-
Port-forward to access the ArgoCD server:
-
Access the ArgoCD UI at:
https://localhost:8080
-
Get the initial admin password:
-
Login with username
admin
and the password retrieved above.
Grafana
Grafana is deployed as part of the helicone-monitoring component and provides observability dashboards for monitoring your Helicone deployment. It works alongside Prometheus to collect and visualize metrics from all your services.
Accessing Grafana UI
-
Port-forward to access the Grafana server:
-
Access the Grafana UI at:
http://localhost:3000
-
Get the admin password (if using default configuration):
-
Login with username
admin
and the password retrieved above. -
Pre-configured dashboards for Helicone services should be available under the Dashboards section.
Configuring S3 (Optional)
Terraform Setup
Go to terraform/s3, then terraform validate
followed by terraform apply
Manual Setup
If minio is enabled, then it will take the place of S3. Minio is a storage solution similar to AWS S3, which can be used for local testing. If minio is disabled by setting the enabled flag under that service to false, then the following parameters are used to configure the bucket:
- s3BucketName
- s3Endpoint
- s3AccessKey (secret)
- s3SecretKey (secret)
Make sure to enable the following CORS policy on the S3 bucket, such that the web service can fetch URL’s from the bucket. To do so in AWS, in the bucket settings, set the following under Permissions -> Cross-origin resource sharing (CORS):
Aurora Setup via Terraform
To set up an Aurora postgresql database using Terraform, follow these steps:
-
Navigate to the terraform/aurora directory:
-
Initialize Terraform:
-
Validate the Terraform configuration:
-
Apply the Terraform configuration to create the Aurora cluster:
After the aurora resource is created, make sure to set enabled to false for postgresql. This will allow the aurora cluster to be used in its place.