# Modal Projects at AI Tinkerers

> Canonical HTML: https://aitinkerers.org/technologies/modal
> Markdown URL: https://aitinkerers.org/technologies/modal.md
> Technology record last updated: 2026-09-18T15:13:39Z
> Generated: 2026-09-22T16:38:48Z

Modal is the unified cloud platform for data and AI, providing elastic, serverless infrastructure to instantly run and deploy any Python code, from zero to thousands of GPUs.

Modal delivers high-performance, developer-focused cloud infrastructure for data and AI workloads (LLM fine-tuning, Generative AI inference, computational biotech). The platform is built from the ground up: it features a custom file system, container runtime, and orchestration engine, rejecting standard solutions like Docker and Kubernetes to achieve near-instant boot times. This architecture enables elastic GPU scaling, letting you scale from zero to thousands of CPUs or GPUs in seconds. Pricing is strictly usage-based, ensuring you only pay for the compute time your code is actively running, eliminating fixed cluster costs and capacity planning headaches.

- Official technology site: https://modal.com
- Public AI Tinkerers demos and talks: 14
- Result page: 1 of 1

## Recent Public Talks and Demos

### [Create Your Own DeepSeek Moment](https://nyc.aitinkerers.org/talks/rsvp_UswbXaI7O_0)

I built an autonomous loop that tests and proves small models (finetuned in some cases) can replace large ones on real traffic before safely rerouting to them. Live I'll show the decision engine choosing which work to optimize, fine-tuning candidate models, clustering traffic to anonymize data, and if time allows examples of a small model beating a large one on specialized work, an eval that fails and keeps the large model live, and the prove-gate thresholds that make it production-safe.

- Event context: August Demo Day ft Runpod, Veris, Openrouter, — 2026-08-19 — New York City
- Public talk page: https://nyc.aitinkerers.org/talks/rsvp_UswbXaI7O_0

### [Watch 1 hour highly techincal YouTubes in 5 minutes with AI!](https://seattle.aitinkerers.org/talks/rsvp_cjwx88z4AkE)

AG is an agent that watches YouTube podcasts for you so you know which ones to really dig into. With AG, see in 5 minutes a summary of the YouTube, key quotes, see key blackboard / slide / code sections, jump around key passages, and decide if you should spend the full time on the video. Break down highly techincal episodes from Dwarkesh, Lenny, AI Engineer, and more!

- Event context: AI Tinkerers Seattle Summer Bash — 2026-07-29 — Seattle
- Public talk page: https://seattle.aitinkerers.org/talks/rsvp_cjwx88z4AkE

### [Speak mk1: A multimodal mamba-attention hybrid model for speech therapy](https://dubai.aitinkerers.org/talks/rsvp_f9s0FLsNTT0)

I am currently building a mamba-attention hybrid encoder and decoder from scratch for speech therapy, not fine-tuning, not wrapping an API, but training every component end to end on my RTX 4060 laptop. The system has three components I am building in parallel: The first is a custom Mamba SSM-based audio encoder trained on LibriSpeech with multi-task phonological heads for voicing, manner, place of articulation, and correctness detection. The encoder uses a BLIP-2-style Q-Former to bridge audio representations into the language model. The second is a video pipeline using MediaPipe FaceLandmarker to isolate and analyze oral region motion frame by frame, extracting articulatory features like tongue tip position, mouth opening geometry, and lip protrusion in real time from a standard webcam. The third component is SpeakMK1LLM, which serves as the project’s core reasoning engine. To balance cutting-edge architectural research with deployment stability for the current phase, we have developed two iterations of this model: The Hybrid Prototype: A custom 70M parameter Mamba-attention hybrid model designed for high-efficiency sequence modeling. It was trained on a rigorous four-stage curriculum: general pretraining on Tinystories, domain adaptation on CHILDES, clinical knowledge injection from pubmed central, and final instruction tuning. The Deployment Model: A fine-tuned Gemma 3n E4B, which leverages the same four-stage curriculum to adapt its large-scale reasoning capabilities to the specific nuances of Speech-Language Pathology (SLP). The entire stack requires only a webcam and a microphone, which is the point — making clinical-quality articulation feedback accessible without specialized hardware or a therapist present in the room. For the demo I would show the live pipeline taking a child's speech, flagging a specific phoneme error, and generating a graded corrective prompt in real time, alongside the clinician dashboard that logs session data for remote review.

- Event context: AI Tinkerers Dubai - May Demo Day — 2026-05-23 — Dubai
- Public talk page: https://dubai.aitinkerers.org/talks/rsvp_f9s0FLsNTT0

### [VLMs for Electronics Design](https://raleigh.aitinkerers.org/talks/rsvp_PDqUleDH6rI)

An electronics design tool (think KiCAD or Altium) that leverages VLMs to turn vendor datasheet PDFs into live component models extracted directly from manufacturer reference documentation. The AI Tinkerers demo shows how I built the extraction pipeline using small open weight VLMs deployed on Modal, and a UI built on top of Zed's GPUI framework. The code and benchmark results discussed during the talk are now available on Github as the Parselab project.

- Event context: AI Tinkerers Raleigh Meetup — May 6, 2026 — 2026-05-06 — Raleigh
- Public talk page: https://raleigh.aitinkerers.org/talks/rsvp_PDqUleDH6rI

### [Building an end-to-end agentic coding workflow in sandbox](https://seattle.aitinkerers.org/talks/rsvp_A0CI_bsvMlY)

I’ll do a live technical walkthrough of an AI-assisted developer workflow that turns a task into code changes, tests, and a PR-ready diff. I’ll show the actual implementation: task orchestration, sandboxed execution, tool-calling, retry/failure handling, and human-in-the-loop checkpoints. This is a code-only demo (no slides), including what broke, what was hacked together, and what made the workflow reliable enough for daily use.

- Event context: AI Dev Tool Track — Seattle Meetup - April 13th, 2026 — 2026-04-14 — Seattle
- Public talk page: https://seattle.aitinkerers.org/talks/rsvp_A0CI_bsvMlY

### [A playground to run AIFS experiments using serverless inference](https://zurich.aitinkerers.org/talks/rsvp_2Nwv7NahQs4)

The aifs-modal library is a thin serverless wrapper to easily run AIFS inference (for forecasts or reforecasts) on a rented GPU in one Python call, storing the outputs in a versioned Zarr store using Icechunk. The demo shows how (claude and) I wired together anemoi-inference (ECMWF's model runtime), Modal (serverless GPU), and Icechunk (git-like array storage) to get reproducible forecasts or reforecasts from any historical date, all driven from a local Jupyter notebook with no cluster, no Colab, no infrastructure to manage. Initial conditions can be ingested from ECMWF operational analysis or ERA5 reanalysis out of the box. Once the GPU job finishes, the output is managed in a versioned Icechunk store and opens instantly in your local notebook as an xarray Dataset for postprocessing. Ensemble forecasts are also supported: each member runs on a separate GPU in parallel, with members writing concurrently into the same Icechunk store.

- Event context: AI Tinkerers Zurich April 9th — 2026-04-09 — Zürich
- Public talk page: https://zurich.aitinkerers.org/talks/rsvp_2Nwv7NahQs4

### [telli's internal agents infrastructure](https://berlin.aitinkerers.org/talks/rsvp_kNA1GoX6394)

How we provide access to the best coding/debugging agent's to everyone in the company, for : - debugging call issues - finding information about the product - researching bugs - monitoring infra (inspired by https://engineering.ramp.com/post/why-we-built-our-background-agent)

- Event context: AI Tinkerers Berlin Meetup - March 11, 2026 — 2026-03-11 — Berlin
- Public talk page: https://berlin.aitinkerers.org/talks/rsvp_kNA1GoX6394

### [Constrained decoding: Can language models generate cute pixel art?](https://montreal.aitinkerers.org/talks/rsvp_ewFxWIEUn3k)

Recently, I’ve been experimenting with low-level token generation using models like gpt-oss-20b and gemma-3-27b-it. I became particularly interested in constrained decoding, where you apply a mask to the model’s probability distribution at each generation step. That led me to a fun question: what if you constrain the probabilities to a predefined set of “color tokens” and force the model to output them in a grid, essentially generating pixel art? Predictably, the results were… chaotic. But the failures were fascinating, and I’m convinced the limitations were mostly due to my rushed implementation and time constraints.

- Event context: AI Tinkerers Montreal: Demo Night — November 20, 2025 — 2025-11-20 — Montreal
- Public talk page: https://montreal.aitinkerers.org/talks/rsvp_ewFxWIEUn3k

### [Personalized Product Discovery with Agentic AI: Beyond Traditional Recommendation Engines](https://singapore.aitinkerers.org/talks/rsvp_TYi5dO1vY58)

E-commerce platforms have long used recommendation engines to boost sales. Yet, these systems often fall short when it comes to true personalization. This talk introduces how Agentic AI can revolutionize recommendations for mobiles, gadgets, accessories, and insurance plans—shifting from static algorithms to adaptive, goal-oriented agents. We’ll discuss how these agents combine customer intent, multi-modal data, and conversational interactions to deliver next-generation recommendations that drive engagement, increase trust, and create measurable business impact.

- Event context: AIT Singapore - AI Agents Showcase - 7th Oct 2025 — 2025-10-07 — Singapore
- Public talk page: https://singapore.aitinkerers.org/talks/rsvp_TYi5dO1vY58

### [SandboxJS: Terraform for AI Sandboxes](https://sf.aitinkerers.org/talks/rsvp_sUu99YMS6gU)

I created an open source library for quickly building agents on any AI sandbox provider. It abstracts the basic functions of a sandbox (running code, building from a Dockerfile, etc.) for building computer use agents, coding agents, and reasoning agents. I'll also share my thoughts on the world of open source code sandboxes based on my contributions at E2B, Daytona, and Cua.

- Event context: AI Infra Jam: Demos That Weren’t Possible Two Months Ago — 2025-09-25 — San Francisco
- Public talk page: https://sf.aitinkerers.org/talks/rsvp_sUu99YMS6gU

### [Learnings from building AI agents](https://london.aitinkerers.org/talks/rsvp_uF6QxcY_VTE)

Quick Summary: Learnings from Building AI Agents for Code Review We built an AI code review agent for cubic that initially produced too many noisy, low-value comments—making code reviews harder instead of easier. After several architecture revisions, we cut false positives by 51% without losing important feedback. I'll explain how

- Event context: AI Tinkerers London Meetup - July 16th, 2025 — 2025-07-16 — London
- Public talk page: https://london.aitinkerers.org/talks/rsvp_uF6QxcY_VTE

### [Language Driven Organ-Lesion Predictions in a Large Scale Toxicological Experiment](https://milan.aitinkerers.org/talks/rsvp_SO2HacjeF2k)

Predicting compound toxicity relies predominantly on representing molecules at various levels of detail, with evaluation based on extensive experimental studies. These studies test a vast array of factors to statistically estimate the potentially harmful outcomes of treatment conditions. This research analyzes toxicology experiments from a new perspective. Rather than studying experimental factors separately or developing specialized deep learning models, this approach interprets experimental factors as strings and uses general-purpose pretrained language models for surprisingly accurate predictions. Textual descriptors are either projected into a dense vector space with embedding models and then integrated into a nested cross-validation pipeline, or given as input to state-of-the-art Large Language Models (LLMs) that directly attempt zero-shot classification using schema-constrained generation. The experimental validation uses data from the repeated-dose Open TG-GATEs dataset, which exposes the same rat clone to 142 compounds over four different time periods at three levels of dosage and contains histopathology annotations for both kidney and liver lesions.

- Event context: AI Tinkerers Milan - June 10, 2025 - Community Demos & Networking — 2025-06-10 — Milan
- Public talk page: https://milan.aitinkerers.org/talks/rsvp_SO2HacjeF2k

### [Wrapperone](https://rome.aitinkerers.org/talks/rsvp_zsIccEq3MtM)

A technical library designed for power users who need: Parallel, rate-limited request processing to LLM providers (OpenAI, Anthropic, VLLM, LiteLLM, and OpenRouter). A system to store conversation states (chat threads) with built-in immutability and versioning. Automatic tool integration (both function-based and schema-based). Flexible response formats: text, JSON, structured schemas, or multi-step workflows. The core design revolves around Entities. Each conversation, message, or piece of configuration is an Entity that undergoes forking and versioning whenever modified. This ensures you have a complete lineage of conversation states without accidental in-place mutations.

- Event context: AI Tinkerers Rome – March 3, 2025 — 2025-03-03 — Rome
- Public talk page: https://rome.aitinkerers.org/talks/rsvp_zsIccEq3MtM

### [Chatting With Logs -- Rethinking the Log Search Interface](https://atlanta.aitinkerers.org/talks/rsvp_7hVDp4YESNo)

Most of the observability data products have different languages that the developers have to learn. Oftentimes, these languages are significantly different from each other and moving for a developer to move between products, it would involve a big learning curve. In our work, we are looking at making these query interfaces easier by finetuning LLMs to generate these languages. This involves a set of challenges as the logs often do not fit into the context window of most LLMs, and off the shelf LLMs lack application specific knowledge for generating the queries. In this demo, I'd walk through the framework that organizations would need to follow for finetuning their own models for this task and deploy it into production. Along with this, I will showcase the dataset, finetuned models and a demo currently deployed using Modal labs.

- Event context: AI Tinkerers Atlanta January Meetup — 2025-01-23 — Atlanta
- Public talk page: https://atlanta.aitinkerers.org/talks/rsvp_7hVDp4YESNo

## Related Technologies

- [vLLM](https://aitinkerers.org/technologies/vllm) ([Markdown](https://aitinkerers.org/technologies/vllm.md)) — 33 public demos
- [BERT](https://aitinkerers.org/technologies/bert) ([Markdown](https://aitinkerers.org/technologies/bert.md)) — 179 public demos
- [BLOOM](https://aitinkerers.org/technologies/bloom) ([Markdown](https://aitinkerers.org/technologies/bloom.md)) — 115 public demos
- [Claude](https://aitinkerers.org/technologies/claude) ([Markdown](https://aitinkerers.org/technologies/claude.md)) — 174 public demos
- [GPT-3](https://aitinkerers.org/technologies/gpt-3) ([Markdown](https://aitinkerers.org/technologies/gpt-3.md)) — 191 public demos
- [GPT-4](https://aitinkerers.org/technologies/gpt-4) ([Markdown](https://aitinkerers.org/technologies/gpt-4.md)) — 529 public demos
- [Llama-2](https://aitinkerers.org/technologies/llama-2) ([Markdown](https://aitinkerers.org/technologies/llama-2.md)) — 227 public demos
- [Node](https://aitinkerers.org/technologies/node) ([Markdown](https://aitinkerers.org/technologies/node.md)) — 99 public demos
- [PaLM 2](https://aitinkerers.org/technologies/palm-2) ([Markdown](https://aitinkerers.org/technologies/palm-2.md)) — 116 public demos
- [Python](https://aitinkerers.org/technologies/python) ([Markdown](https://aitinkerers.org/technologies/python.md)) — 662 public demos
- [RoBERTa](https://aitinkerers.org/technologies/roberta) ([Markdown](https://aitinkerers.org/technologies/roberta.md)) — 118 public demos
- [AgentGPT](https://aitinkerers.org/technologies/agentgpt) ([Markdown](https://aitinkerers.org/technologies/agentgpt.md)) — 3 public demos
- [Agentic AI frameworks](https://aitinkerers.org/technologies/agentic-ai-frameworks) ([Markdown](https://aitinkerers.org/technologies/agentic-ai-frameworks.md)) — 1 public demo
- [AGiXT](https://aitinkerers.org/technologies/agixt) ([Markdown](https://aitinkerers.org/technologies/agixt.md)) — 2 public demos
- [AIFS](https://aitinkerers.org/technologies/aifs) ([Markdown](https://aitinkerers.org/technologies/aifs.md)) — 1 public demo
- [Amazon EventBridge](https://aitinkerers.org/technologies/amazon-eventbridge) ([Markdown](https://aitinkerers.org/technologies/amazon-eventbridge.md)) — 1 public demo
- [anemoi-inference](https://aitinkerers.org/technologies/anemoi-inference) ([Markdown](https://aitinkerers.org/technologies/anemoi-inference.md)) — 1 public demo
- [Anthropic](https://aitinkerers.org/technologies/anthropic) ([Markdown](https://aitinkerers.org/technologies/anthropic.md)) — 36 public demos