You can use Helicone with your OpenAI compatible models that are deployed on Anyscale. Follow the Helicone integration as normal in the proxy approach but add the following header. 
Helicone-OpenAI-API-Base: https://api.endpoints.anyscale.com/v1
This will route traffic through Helicone to your Anyscale deployment.