Groq SDK

The Groq SDK provides high-speed access to the LPU Inference Engine for sub-second LLM responses.

Engineered for extreme low-latency performance, the Groq SDK allows developers to integrate models like Llama 3 and Mixtral 8x7B into production environments with minimal overhead. It mirrors the OpenAI API schema (standardizing integration) while leveraging Groq's LPU architecture to hit throughput speeds exceeding 800 tokens per second. Use the Python or Node.js packages to manage chat completions, streaming, and tool use with a few lines of code.

https://github.com/groq/groq-python