Anyscale Endpoints

TLDR: Anyscale releases Anyscale Endpoints Preview for LLM developers to run and and fine-tune open-source LLMs fast, cost-efficiently, and at scale. Get started now.

Generative AI and Large Language Models (LLMs) have propelled the AI domain to new heights in recent times. However, along with the evolution and capabilities of these sophisticated models, numerous challenges arise when it comes to their deployment and fine-tuning. Recognizing these hurdles, Anyscale is releasing a preview of Anyscale Endpoints to help developers integrate fast, cost-efficient, and scalable LLM APIs.

Here’s a quick glimpse at what Anyscale Endpoints offers:

• State-of-the-art open-source and proprietary performance and cost optimizations.
• A serverless approach to running open-source LLM models, mitigating infrastructure complexities.
• A seamless transition for running base or fine-tuned models on your cloud.
• Streaming response.
• Utilization of the power of Ray Serve and Ray Train libraries.
• Integration within your workflow is straightforward, all you have to do is sign up in less than two minutes.
• Compatibility with the OpenAI API and SDK enables a smooth integration of Anyscale LLM Endpoints with minimal code changes.