# LiteLLM Projects at AI Tinkerers

> Canonical HTML: https://aitinkerers.org/technologies/litellm
> Markdown URL: https://aitinkerers.org/technologies/litellm.md
> Technology record last updated: 2026-02-24T03:36:40Z
> Generated: 2026-09-22T22:33:59Z

LiteLLM is the unified LLM gateway: call 100+ models (OpenAI, Anthropic, Azure, etc.) using a single, standardized OpenAI-compatible API.

LiteLLM acts as your production-grade LLM gateway, simplifying complex multi-model deployments. It unifies over 100 LLM providers—including OpenAI, Anthropic, and VertexAI—under a single, consistent API call structure (the OpenAI format). This standardization eliminates SDK friction. Key features include the LiteLLM Router for automatic retry and fallback logic across deployments, ensuring high reliability. Additionally, the Proxy Server centralizes cost tracking, allows granular budget setting per virtual key, and provides load balancing, making it essential for ML Platform teams managing scalable, cost-optimized Gen AI applications.

- Official technology site: https://litellm.ai
- Public AI Tinkerers demos and talks: 21
- Result page: 1 of 1

## Recent Public Talks and Demos

### [Run your autonomous business like crowd control at music festivals](https://durango.aitinkerers.org/talks/rsvp_l-EjDuk3piI)

Using Paperclip primitives for cases, pipelines, agents and remote runtimes I have (re?)built most of my dark factory for product building, and merge + rollout. Instead of letting the agents roam the field free, I established items (merch, food, beer) and crowd control (lines, queues) along with workers to help merch get signed, food finished properly and cold beers poured. Unlike the music festival, I also established high fidelity "more than just text" agents/visualizations to better observe and collaborate with the system.

- Event context: AI Tinkerers Durango — September Lunch Builder Meetup — 2026-09-15 — Durango
- Public talk page: https://durango.aitinkerers.org/talks/rsvp_l-EjDuk3piI

### [LMStack - Development stack using local LLMs](https://seattle.aitinkerers.org/talks/rsvp_A8GvI03RSK8)

Writing real code with local LLMs is difficult. lmstack is a local stack for real development which puts your GPU to work on the right kind of tasks. LMStack introduces an infrastructure manager and a task classifier. Both of them work together to make sure local models are always at work. I've been using this stack for a month to build features on an open source project - Clawrium. LMStack show why the entire stack needs to be different when using local llms and what're the pitfalls of using standard tooling. What works and what to avoid.

- Event context: AI Dev Tools Track - Seattle - August 4 — 2026-08-05 — Seattle
- Public talk page: https://seattle.aitinkerers.org/talks/rsvp_A8GvI03RSK8

### [NeverTwice: cost optimized API's for LLM inference tailored to your personal use case.](https://lausanne.aitinkerers.org/talks/rsvp_oYss-NBmvTQ)

A system capable to identify how you should optimised your queries to keep your standard of accuracy high for a fraction of the costs. The methodology rely heavily on using evals and a set of rules to only evaluate the most promising strategies until you land on the cheapest option for your use case. What I'll show is a little demo + discuss the architecture.

- Event context: AI Tinkerers Lausanne June 2026 Meetup — 2026-06-25 — Lausanne
- Public talk page: https://lausanne.aitinkerers.org/talks/rsvp_oYss-NBmvTQ

### [Agents Building Agents: Reflective Optimization Loops](https://toronto.aitinkerers.org/talks/rsvp_4gS7qartFFk)

I'll show how an AI agent can build and optimize another AI agent, using reflective optimization to find issues, optimize evals, iterate on architecture, find the optimal prompt/model, and more. We've built a system with multiple levels of reflective optimization for agent development. - GEPA: reflective prompt optimization - Synthetic eval generation: going from a 1-off bug to an proper eval you can use in reflective optimization - Expanding reflective optimization beyond prompts: model selection, tool use, subagents -- reflective optimization can drive all levels of agent optimization.

- Event context: AI Tinkerers Toronto - April 2026 - hosted by Shopify — 2026-04-29 — Toronto
- Public talk page: https://toronto.aitinkerers.org/talks/rsvp_4gS7qartFFk

### [Clawloop - the unified learning api](https://zurich.aitinkerers.org/talks/rsvp_sEFw1hfu1rc)

Clawloop is the learning from experience layer for AI agents. It turns traces, failures, and user feedback from production traffic and interactions with training environments into updates to an agent’s harness and weights. The harness includes prompt, memory, and tools, so Clawloop can improve not just the model but also how the agent is instructed and equipped. It plugs into stacks like LiteLLM, OpenClaw, and n8n and connects to training backends like SkyRL and Tinker for weight adaptation.

- Event context: AI Tinkerers Zurich April 9th — 2026-04-09 — Zürich
- Public talk page: https://zurich.aitinkerers.org/talks/rsvp_sEFw1hfu1rc

### [TDLW](https://montreal.aitinkerers.org/talks/rsvp_nT9L3-mZHQU)

a vibe-engineered personal app to help summarize long podcasts on youtube it's still WIP, but by the demo night the UI will be better organized and there will be a RAG implemented so that you could chat about the video with an agent

- Event context: AI Tinkerers Montreal - March Demo Night — 2026-03-24 — Montreal
- Public talk page: https://montreal.aitinkerers.org/talks/rsvp_nT9L3-mZHQU

### [AI That Moves Money: Cashfree’s AI Playbook](https://bengaluru.aitinkerers.org/talks/rsvp_0QSWBvjkAkE)

We’ll present how Cashfree is using AI to automate complex payment flows, and the technical systems that make these AI-driven actions secure, deterministic, and scalable in real-world fintech environments.

- Event context: AI Tinkerers - Bengaluru - November meetup — 2025-11-29 — Bengaluru
- Public talk page: https://bengaluru.aitinkerers.org/talks/rsvp_0QSWBvjkAkE

### [Running a local LLM with guardrails](https://hong-kong.aitinkerers.org/talks/rsvp_C87wbAjEP3o)

started building a local Librechat with SSO, auditing capabilities, and guardrails.

- Event context: AI Tinkerers Hong Kong Meetup - November 27th — 2025-11-27 — Hong Kong
- Public talk page: https://hong-kong.aitinkerers.org/talks/rsvp_C87wbAjEP3o

### [Building an Agentic Orchestrator for LLM Testing and Evaluation](https://berlin.aitinkerers.org/talks/rsvp_jeR5QZF3ucQ)

This talk covers the architecture and implementation of Penelope, the agentic orchestrator used in the Rhesis framework for testing LLM applications. Penelope acts as a control agent that coordinates multi-step test executions, model calls, and evaluation routines. The session will explain how Penelope manages test definitions, executes adaptive workflows, and interacts with model endpoints via LiteLLM. I will discuss how evaluation tasks are modeled as agent goals, how results are captured in structured form, and how the system supports reproducible multi-turn tests. We will also look at the interface between the orchestration layer and the evaluation layer, including how LLMs are used to generate test cases, expected behaviors, and automatic scoring prompts.

- Event context: AI Tinkerers Berlin Meetup - November 12th, 2025 — 2025-11-12 — Berlin
- Public talk page: https://berlin.aitinkerers.org/talks/rsvp_jeR5QZF3ucQ

### [Automating error analysis - what works and what doesn't](https://liverpool.aitinkerers.org/talks/rsvp_9LjshIDL1Do)

Error analysis is known to be the highest ROI step in building reliable agents that work. However, as agent & task complexity grow, manual approaches to it become prohibitive, and naive ways of automating it remain unsatisfying. How *should* developers scale up this key step in agent evaluation? Here, I'll present our recent findings on what works - and what doesn't - when automating error analysis. TL;DR: we can do a lot better than stuffing everything into an LLM, with a bit of intentional context engineering and task decomposition.

- Event context: AI Tinkerers Liverpool HITL Meetup at FACT Lab — 2025-10-28 — Liverpool
- Public talk page: https://liverpool.aitinkerers.org/talks/rsvp_9LjshIDL1Do

### [Who Watches The Watchbots?](https://seattle.aitinkerers.org/talks/rsvp_SJ_UH0RNInc)

We’re building a local-first open-source tool to make AI Assistants behave. AI Control is a new, empirical field pioneered by Redwood Research focused on mitigating potentially harmful AI actions. I’m not affiliated with Redwood, but I know them and I’ve worked down the hall from them. Their focus is on future “scheming” frontier AI, but the approaches they’ve developed and tested can also be applied to keeping prosaic AI systems in check. We’re building a system to make it easy to implement empirically-verified AI Control strategies as well as any other intervention you can imagine locally, so you can automate things like “making sure the AI isn’t bypassing tests” or “require explicit human confirmation with a big obvious warning sign before executing dangerous or suspicious-seeming tool calls”. We’re building on top of LiteLLM to make it easy to deploy this for virtually any LLM-backed system.

- Event context: October Meetup - Science Fair at Foundations — 2025-10-23 — Seattle
- Public talk page: https://seattle.aitinkerers.org/talks/rsvp_SJ_UH0RNInc

### [No Data for the Witch’s Cauldron: Using Safe, Open-Source AI Chats systems](https://montreal.aitinkerers.org/talks/rsvp_bNbMDKHWgAc)

Stop building chat applications, reuse existing ones and extend them ! Sharing a technical journey to use a safe and compliant solution for chatting with private project data, leveraging local and remote LLMs, internal RAG systems, MCP servers, live voice, image generation, web search, and a code interpreter. The result is similar to ChatGPT but built entirely on open-source technologies and trusted cloud providers LLM APIs. Using OpenWeb UI as the chat front-end, connected to a foundation model on Azure AI and integrated with internal RAG system for documentation. LiteLLM serves as the OpenAPI layer for both OpenWeb UI and server calls, supporting claude code, codex-cli, and a custom workaround for gemini-cli, as we operate exclusively on AWS Bedrock and Azure AI.

- Event context: AI Tinkerers Montreal – Spooky October 2025 Meetup — 2025-10-21 — Montreal
- Public talk page: https://montreal.aitinkerers.org/talks/rsvp_bNbMDKHWgAc

### [Who Watches The Watchbots?](https://seattle.aitinkerers.org/talks/rsvp_H3oWhromOkY)

We’re building a local-first open-source tool to make AI Assistants behave. AI Control is a new, empirical field pioneered by Redwood Research focused on mitigating potentially harmful AI actions. I’m not affiliated with Redwood, but I know them and I’ve worked down the hall from them. Their focus is on future “scheming” frontier AI, but the approaches they’ve developed and tested can also be applied to keeping prosaic AI systems in check. We’re building a system to make it easy to implement empirically-verified AI Control strategies as well as any other intervention you can imagine locally, so you can automate things like “making sure the AI isn’t bypassing tests” or “require explicit human confirmation with a big obvious warning sign before executing dangerous or suspicious-seeming tool calls”. We’re building on top of LiteLLM to make it easy to deploy this for virtually any LLM-backed system.

- Event context: AI Tinkerers Seattle September Meetup — September 30, 2025 — 2025-10-01 — Seattle
- Public talk page: https://seattle.aitinkerers.org/talks/rsvp_H3oWhromOkY

### [Standard Agent](https://dublin.aitinkerers.org/talks/rsvp_VAiiOX7UQEE)

Hello hello, I would like to give a shot at presenting something called a Standard Agent which is an open source agent library. LLMs are hard. Tooling is hard. This is an attempt at showing maybe what sticks them together need not always be hard. It’s less than 1000 lines of core code, doesn’t try to be magic, and is ready to be cloned, forked, extended, or replaced.

- Event context: AI Tinkerers Dublin Anniversary Meetup — September 10, 2025 — 2025-09-10 — Dublin
- Public talk page: https://dublin.aitinkerers.org/talks/rsvp_VAiiOX7UQEE

### [How to build large-scale, deterministic, high-quality conversational workflows with small/free models](https://houston.aitinkerers.org/talks/rsvp_UIqfUoxe5Gw)

This talk will go over the challenges of intent detection, parameter extraction and human-in-the-loop interaction for conversational workflows and how to solve them in the context of building co-pilots for existing applications. We will then demonstrate how to build non-trivial AI co-pilots for such workflows using an open-source framework called fastWorkflow. Finally, we will review the performance of fastworkflow vs. large models on the Tau Bench retail workflow to additionally demonstrate its effectiveness as an agentic platform.

- Event context: AI Tinkerers Houston Meetup — September 9, 2025 — 2025-09-09 — Houston
- Public talk page: https://houston.aitinkerers.org/talks/rsvp_UIqfUoxe5Gw

### [AutoGrading Student Answers using](https://dubai.aitinkerers.org/talks/rsvp_nZAAyR1_6s0)

I implemented a full scale autograding solution for written and spoken answers for 1million userbasr 25k concurrent users. We used whisper and llama. On different providers to meet the scale.

- Event context: AI Tinkerers Dubai Meetup — August 2025 Demo Day — 2025-08-23 — Dubai
- Public talk page: https://dubai.aitinkerers.org/talks/rsvp_nZAAyR1_6s0

### [How to use Qwen3-Coder (or any other LLM) with Claude Code](https://munich.aitinkerers.org/talks/rsvp_tpX-SXAb2Io)

A simple way for Claude Code users to switch from the costly Claude models to the newly released SOTA open-source/weights coding model, Qwen3-Coder, or any other LLM, locally or online.

- Event context: AI Tinkerers Munich - July 25 — 2025-07-25 — Munich
- Public talk page: https://munich.aitinkerers.org/talks/rsvp_tpX-SXAb2Io

### [Cognitive Architecture + Conversational Agents](https://seattle.aitinkerers.org/talks/rsvp_WBCt9jEiih4)

I'll demo how I've applied cognitive architecture in the form of utterance classification and modular memory to create a highly configurable deterministic layer for AI based chat interactions

- Event context: AI Tinkerers Seattle – June Meetup — 2025-06-28 — Seattle
- Public talk page: https://seattle.aitinkerers.org/talks/rsvp_WBCt9jEiih4

### [School Quiz Grader with DocRouter.AI](https://boston.aitinkerers.org/talks/rsvp_9vvWQ2XeaSc)

DocRouter.AI can serve as the AI tech stack backbone for companies in HealthTech, Accounting, Legal, Insurance verticals dealing with unstructured documents. You customize your application as needed for your use case - while relying on DocRouter.AI for full power of LLM prompt engineering and human-in-the-loop as back end. In this demo, we have an EdTech application: An AI evaluation tool for 5th grade quizzes, with open form items (no multiple choice!), with the teacher-in-the-loop.

- Event context: AI Tinkerers Boston May Meetup — 2025-05-20 — Boston
- Public talk page: https://boston.aitinkerers.org/talks/rsvp_9vvWQ2XeaSc

### [MCP Servers vs. Message Bus Architecture for AI Agents](https://las-vegas.aitinkerers.org/talks/rsvp_i3AInAf5ez8)

I have a brief outline at https://github.com/johnsmith968530/mcp_vs_redis_streams/blob/main/README.md

- Event context: AI Tinkerers Las Vegas - May Inaugural Meetup — 2025-05-09 — Las Vegas
- Public talk page: https://las-vegas.aitinkerers.org/talks/rsvp_i3AInAf5ez8

### [Utilizing LiteLLM for multi-cloud / mutli-llm deployments](https://portland.aitinkerers.org/talks/rsvp_239P2zw386E)

I'll give a brief description of LiteLLM and how we're using it at CaseMark as well as the problems it's solved for us

- Event context: AI Tinkerers Portland Inaugural Meetup - July — 2024-07-24 — Portland
- Public talk page: https://portland.aitinkerers.org/talks/rsvp_239P2zw386E

## Related Technologies

- [FastAPI](https://aitinkerers.org/technologies/fastapi) ([Markdown](https://aitinkerers.org/technologies/fastapi.md)) — 181 public demos
- [OpenRouter](https://aitinkerers.org/technologies/openrouter) ([Markdown](https://aitinkerers.org/technologies/openrouter.md)) — 34 public demos
- [PostgreSQL](https://aitinkerers.org/technologies/postgresql) ([Markdown](https://aitinkerers.org/technologies/postgresql.md)) — 106 public demos
- [Python](https://aitinkerers.org/technologies/python) ([Markdown](https://aitinkerers.org/technologies/python.md)) — 662 public demos
- [Docker](https://aitinkerers.org/technologies/docker) ([Markdown](https://aitinkerers.org/technologies/docker.md)) — 147 public demos
- [MCP](https://aitinkerers.org/technologies/mcp) ([Markdown](https://aitinkerers.org/technologies/mcp.md)) — 129 public demos
- [Ollama](https://aitinkerers.org/technologies/ollama) ([Markdown](https://aitinkerers.org/technologies/ollama.md)) — 77 public demos
- [Claude](https://aitinkerers.org/technologies/claude) ([Markdown](https://aitinkerers.org/technologies/claude.md)) — 174 public demos
- [Claude Code](https://aitinkerers.org/technologies/claude-code) ([Markdown](https://aitinkerers.org/technologies/claude-code.md)) — 216 public demos
- [Pydantic](https://aitinkerers.org/technologies/pydantic) ([Markdown](https://aitinkerers.org/technologies/pydantic.md)) — 30 public demos
- [Agent Skills](https://aitinkerers.org/technologies/agent-skills) ([Markdown](https://aitinkerers.org/technologies/agent-skills.md)) — 4 public demos
- [Amazon Bedrock](https://aitinkerers.org/technologies/amazon-bedrock) ([Markdown](https://aitinkerers.org/technologies/amazon-bedrock.md)) — 20 public demos
- [AWS Bedrock](https://aitinkerers.org/technologies/aws-bedrock) ([Markdown](https://aitinkerers.org/technologies/aws-bedrock.md)) — 9 public demos
- [DistilBERT](https://aitinkerers.org/technologies/distilbert) ([Markdown](https://aitinkerers.org/technologies/distilbert.md)) — 2 public demos
- [Evals](https://aitinkerers.org/technologies/evals) ([Markdown](https://aitinkerers.org/technologies/evals.md)) — 4 public demos
- [GEPA](https://aitinkerers.org/technologies/gepa) ([Markdown](https://aitinkerers.org/technologies/gepa.md)) — 3 public demos
- [GLM](https://aitinkerers.org/technologies/glm) ([Markdown](https://aitinkerers.org/technologies/glm.md)) — 3 public demos
- [GLM 5](https://aitinkerers.org/technologies/glm-5) ([Markdown](https://aitinkerers.org/technologies/glm-5.md)) — 1 public demo