# LoRA Projects at AI Tinkerers

> Canonical HTML: https://aitinkerers.org/technologies/lora
> Markdown URL: https://aitinkerers.org/technologies/lora.md
> Technology record last updated: 2026-09-18T15:13:58Z
> Generated: 2026-09-22T07:36:31Z

LoRA (Low-Rank Adaptation) is a Parameter-Efficient Fine-Tuning (PEFT) technique that accelerates LLM training by injecting small, trainable rank decomposition matrices into the frozen pre-trained weights.

LoRA is the premier solution for adapting Large Language Models (LLMs) without the prohibitive cost of full fine-tuning. The method freezes the original model weights and injects lightweight, low-rank matrices (A and B) into the Transformer layers, specifically targeting attention weights (Wq and Wv). This approach delivers massive resource savings: for a model like GPT-3 175B, LoRA reduces the number of trainable parameters by up to 10,000 times and cuts GPU memory requirements by 3 times. Crucially, its linear design allows the new matrices to be merged with the original weights upon deployment, ensuring zero additional inference latency.

- Official technology site: https://arxiv.org/abs/2106.09685
- Public AI Tinkerers demos and talks: 17
- Result page: 1 of 1

## Recent Public Talks and Demos

### [The loss curve lied: catching hidden safety drift inside the fine-tuning, automated with an agent!](https://zurich.aitinkerers.org/talks/rsvp_Vnt244xXFE0)

We built a fine-tuning where a Claude-Code based agent that helps founders and researchers fine-tune models on their datasets, and get more insights than just the training and eval losses. In particular, we also show the model’s regress on safety and general capability, metrics that otherwise would be overlooked. For example, we give it a medical instruction dataset; the agent asks a few multiple-choice questions and proposes concepts like "recommends unsafe or contraindicated actions," "downplays red-flag symptoms that need urgent care," and "implies clinical authority it doesn't have," then writes the run config. Then, for each concept, it fits a single direction in the model's activations (difference-of-means, following Persona Vectors, Chen et al. 2025, arXiv:2507.21509) that “encodes” that concept, and reads that direction off every checkpoint during LoRA training. Live we can show the agent conversation proposing the concepts, as how they drift alongside per-checkpoint evals. This is part of an ongoing research agenda with a collaboration of researchers from ETH Zurich, ETH AI Center and Agentic Systems Lab. We formulated our research into an agentic harness and live demo in an anthropic hackathon last week, and want to share more about it! Also, we imagine this as an open-source effort.

- Event context: AI Tinkerers Zürich: July Build Night with Swisscom Ventures — 2026-07-01 — Zürich
- Public talk page: https://zurich.aitinkerers.org/talks/rsvp_Vnt244xXFE0

### [Speak mk1: A multimodal mamba-attention hybrid model for speech therapy](https://dubai.aitinkerers.org/talks/rsvp_f9s0FLsNTT0)

I am currently building a mamba-attention hybrid encoder and decoder from scratch for speech therapy, not fine-tuning, not wrapping an API, but training every component end to end on my RTX 4060 laptop. The system has three components I am building in parallel: The first is a custom Mamba SSM-based audio encoder trained on LibriSpeech with multi-task phonological heads for voicing, manner, place of articulation, and correctness detection. The encoder uses a BLIP-2-style Q-Former to bridge audio representations into the language model. The second is a video pipeline using MediaPipe FaceLandmarker to isolate and analyze oral region motion frame by frame, extracting articulatory features like tongue tip position, mouth opening geometry, and lip protrusion in real time from a standard webcam. The third component is SpeakMK1LLM, which serves as the project’s core reasoning engine. To balance cutting-edge architectural research with deployment stability for the current phase, we have developed two iterations of this model: The Hybrid Prototype: A custom 70M parameter Mamba-attention hybrid model designed for high-efficiency sequence modeling. It was trained on a rigorous four-stage curriculum: general pretraining on Tinystories, domain adaptation on CHILDES, clinical knowledge injection from pubmed central, and final instruction tuning. The Deployment Model: A fine-tuned Gemma 3n E4B, which leverages the same four-stage curriculum to adapt its large-scale reasoning capabilities to the specific nuances of Speech-Language Pathology (SLP). The entire stack requires only a webcam and a microphone, which is the point — making clinical-quality articulation feedback accessible without specialized hardware or a therapist present in the room. For the demo I would show the live pipeline taking a child's speech, flagging a specific phoneme error, and generating a graded corrective prompt in real time, alongside the clinician dashboard that logs session data for remote review.

- Event context: AI Tinkerers Dubai - May Demo Day — 2026-05-23 — Dubai
- Public talk page: https://dubai.aitinkerers.org/talks/rsvp_f9s0FLsNTT0

### [MARSYS: Multi-Agent Workflows Without the Plumbing](https://lausanne.aitinkerers.org/talks/rsvp_eigQG2pH8qI)

MARSYS is an open-source Python framework for building multi-agent workflows. Define your agents and who talks to who -- the framework handles parallel execution, branch isolation, convergence, context management, and routing automatically. It works with 7+ LLM providers out of the box (OpenAI, Anthropic, Google, local models), ships with ready-to-use agents (browser, file operations, code executor, data analyst), and lets you fine-tune local models directly from execution traces. `pip install marsys` and you're running.

- Event context: AI Tinkerers Lausanne April 2026 Meetup — 2026-04-30 — Lausanne
- Public talk page: https://lausanne.aitinkerers.org/talks/rsvp_eigQG2pH8qI

### [\[UofT\] Print Anything: Building the Pipeline from Natural Language to 3D Object](https://toronto.aitinkerers.org/talks/rsvp_z0pA85I3o0k)

What if you could 3D print something simply by describing it in plain English, just as you ask ChatGPT a question? That is the idea behind our project. Our system fine-tunes a Qwen 3 model with LoRA in Google Colab to generate ImplicitCAD code from natural language prompts. The generated code is then compiled into STL files, which can be used for 3D printing. ImplicitCAD is a code-based CAD tool, similar to OpenSCAD. Many people in the 3D printing community like it because script-based design allows precise and reproducible measurements. However, the downside is that users need to know how to write code. To address this, we built a two-phase training pipeline. In the first phase, the model learns ImplicitCAD syntax using real examples collected and filtered from the Thingiverse dataset on HuggingFace. This helps the model understand how 3D geometry is written in code. In the second phase, we introduce logical and spatial reasoning problems so the model can go beyond simple pattern matching. In addition to the model, we also built a web-based preview environment and a Dockerized version of ImplicitCAD. This removes the usual edit-compile-wait workflow that makes these tools difficult to use. The project is meant to be a starting point that future teams can build on and extend.

- Event context: AI Tinkerers Toronto - March - hosted by Mozilla! — 2026-03-25 — Toronto
- Public talk page: https://toronto.aitinkerers.org/talks/rsvp_z0pA85I3o0k

### [Continuously Learning AI in Regulated Environments: Versioned Models, Secure Execution, and Controlled Adaptation](https://milan.aitinkerers.org/talks/rsvp_cCBE4KuVMmU)

Regulated industries face a fundamental tension: AI systems need to adapt and improve, yet compliance requires reproducibility, auditability, and strict execution controls. Hyprstream addresses this challenge by treating models as versioned, Git-native artifacts. Base models and fine-tuned adapters are stored as repositories, enabling branching, commit history, and reproducible rollbacks. Every model state becomes traceable. Every adaptation becomes auditable. Within a unified runtime, Hyprstream supports both inference serving and LoRA-based fine-tuning, allowing systems to learn from domain-specific data while remaining inside controlled infrastructure. Model branches can be deployed as distinct OpenAI-compatible endpoints, enabling parallel validation, staged rollouts, and controlled promotion to production. Security is layered by design: Transport encryption and signed RPC envelopes Policy-based authorization Isolated execution via microVM-backed workers Local-first deployment without mandatory SaaS dependencies The result is an architecture that supports continuously learning applications while meeting the demands of compliance, traceability, and operational control. This talk explores how AI systems can evolve without sacrificing governance — and how regulated environments can adopt adaptive AI without relinquishing oversight.

- Event context: AI Tinkerers Milan - February 24, 2026 - Agentic Orchestration in Financial Services: Architectures & Demos — 2026-02-24 — Milan
- Public talk page: https://milan.aitinkerers.org/talks/rsvp_cCBE4KuVMmU

### [Extracting RFC 5545 RRULE Compliant Schedule Data in valid JSON with only 0.6B Parameters](https://seattle.aitinkerers.org/talks/rsvp_DaYlLeV25ZU)

We have a vast amount of unstructured schedule data on community services available to communities across 20 states. It is easier for data managers and community members to write brief notes, though service delivery schedules can be quite complex when translated to data that is interoperable with the iCal standard. This makes it a great task for LLMs... but VRAM/RAM and compute is expensive, you know? I plan to probably show what a row of our data looks like, a community service offered for those in need and its unstructured schedule data. Then, I will take that unstructured data, prompt the model with it, and we can all see if it generated something useful. It will probably all be CLI but I will zoom in my screen ʕ•ᴥ•ʔ

- Event context: AI Tinkerers Seattle: January Meetup — 2026-01-31 — Seattle
- Public talk page: https://seattle.aitinkerers.org/talks/rsvp_DaYlLeV25ZU

### [Teaching LLMs to See: Training a Phi-4 × FastViTHD Vision–Language Model (VLM)](https://seattle.aitinkerers.org/talks/rsvp_Xo7GaoHI-to)

I’ll walk through how I merged Microsoft’s 2.7 B-parameter text-only Phi-4-mini-reasoning LLM with Apple’s high-speed FastViT-HD image encoder to create Friday-VLM—a finetuned Vision Language Model (VLM) that can caption, reason over, and chat about high-resolution images. I’ll cover the end-to-end pipeline (pre-training, instruction fine-tuning, and image encoding) and demo live inference.

- Event context: AI Tinkerers Seattle – June Meetup — 2025-06-28 — Seattle
- Public talk page: https://seattle.aitinkerers.org/talks/rsvp_Xo7GaoHI-to

### [Fine-Tuning Models for Content Moderation with Apple’s MLX Framework](https://orange-county.aitinkerers.org/talks/rsvp_S0EeHEPZUv4)

This talk is a hands-on walkthrough of how to fine-tune large language models—and multi-modal models—locally on Apple-Silicon Macs using Apple’s open-source MLX framework. We’ll: 1. Give a quick MLX overview and why it’s optimized for the M-series GPU/ANE. 2. Explain LoRA / QLoRA and why adapter-based fine-tuning is memory-efficient. 3. Dissect the vision-encoder → adapter → LLM pipeline (SigLIP + Phi-1.5). 4. Show dataset prep in JSONL, CLI commands (mlx_lm.lora, mlx_lm.fuse) and YAML options. 5. Live-interpret training logs, validation curves, and evaluation metrics. 6. End with best-practice checklists for scaling from small experiments to full production runs. Attendees will leave able to replicate the full workflow—dataset → training → evaluation → deployment—entirely on their MacBooks.

- Event context: AI Tinkerers - Orange County Meetup- Wednesday June 4th 2025 — 2025-06-05 — Orange County
- Public talk page: https://orange-county.aitinkerers.org/talks/rsvp_S0EeHEPZUv4

### [Monarch-1: Building Africa-Centric AI](https://nairobi.aitinkerers.org/talks/rsvp_HcIdSj5iSCQ)

I’ll walk through how I took Mistral-7B-Instruct-v0.3 and adapted it into Monarch-1, an AI model specifically tuned for African languages, cultural nuances, and real-world applications. This session will be a deep dive into every step, managing limited GPU resources, handling code-switching in multilingual data, and implementing LoRA-based fine-tuning to keep computational costs low. We’ll also explore the Monarch Benchmark, a custom suite of tests that capture cultural-linguistic nuances and real-world African contexts often missed by mainstream benchmarks. Along the way, I’ll discuss how recent AI developments like the Model Context Protocol (MCP), large context windows, and next-gen multimodal approaches (e.g., Google Gemini2.5 Pro) could extend Monarch-1’s capabilities. Think of it as a code-centric “tell-all” on how to push large language models beyond generic datasets and into localized, ethically grounded AI, no slides, no pitches, just an unfiltered look at building Africa-centric LLMs from scratch.

- Event context: AI Tinkerers - Nairobi Inaugural Meetup (April) — 2025-04-09 — Nairobi
- Public talk page: https://nairobi.aitinkerers.org/talks/rsvp_HcIdSj5iSCQ

### [Generate precise and coherent images on your laptop with Stable Diffudion](https://paris.aitinkerers.org/talks/rsvp_oVGgE1beWr0)

This demo will show how to control poses, styles, and character consistency for use cases like yoga pose visualization (feedback from my ongoing work for my Yoga website www.yogarkana.com) with the local use of Stable Diffusion models and advanced ComfyUI workflows. Learn how to customize Stable Diffusion workflows to meet their your own creative, commercial, or inclusive design needs.

- Event context: AI Tinkerers - Paris Meetup on January 30th — 2025-01-30 — Paris
- Public talk page: https://paris.aitinkerers.org/talks/rsvp_oVGgE1beWr0

### [Diffusion based Interior Customisation for VR Home Tours](https://mumbai.aitinkerers.org/talks/rsvp_mcNaq7cpLUQ)

This project uses a Flux1.dev diffusion model to customise the look of an apartment in a VR Home Tour. The user can pick a style of their choice (Modern, Minimalist, Boho, etc) and the apartment will be styled to match the user's preferences while maintaining the same structure and dimensions. The approach uses Controlnets and LoRAs to create conditioning based on the original apartment layout and then styles the image based on a prompt.

- Event context: AI Tinkerers Mumbai Chapter - 2nd Event Incoming! — 2024-11-23 — Mumbai
- Public talk page: https://mumbai.aitinkerers.org/talks/rsvp_mcNaq7cpLUQ

### [All the Trainingz, No Codez](https://toronto.aitinkerers.org/talks/rsvp_A7iVImRMxy0)

What if it was possible to train, finetune, and preference tune LLMs on your own computer using advanced methods including LoRA, DPO, ORPO, RLHF, etc without writing any code? Today we will demo advanced training of Large Language Models using a home computer using Transformer Lab.

- Event context: AI Tinkerers - Toronto - September 2024 Meetup — 2024-09-20 — Toronto
- Public talk page: https://toronto.aitinkerers.org/talks/rsvp_A7iVImRMxy0

### [Creating Zebra-Llama, an open-source AI rare disease expert](https://toronto.aitinkerers.org/talks/rsvp_p8BY7gZYuAI)

ChatGPT and other LLMs are increasingly being used by rare disease patients and researchers, but this presents significant challenges around the reliability and safety of the responses. Zebra-Llama is an open-source AI model specifically developed for Ehlers-Danlos Syndrome (EDS) expertise. During the demo, I'll discuss how we trained Zebra-Llama on an EDS-specific dataset we built from thousands of research papers and patient experiences to provide accurate responses and cite sources. I’ll walk through the model's architecture, including LoRA fine-tuning of the Llama-3–8b-instruct model, vector database integration with RAG, and use of a Question-Context-Answer format to reduce hallucinations and improve reliability. Finally, I'll show a live demo using the streamlit chat interface. Our team met and developed the model as part of the Rare Disease AI Hackathon organized by Stanford Medicine and Research to the People, and presented it at GitHub headquarters in San Francisco to a distinguished audience including Greg Brockman, President of OpenAI.

- Event context: AI Tinkerers - Toronto - September 2024 Meetup — 2024-09-20 — Toronto
- Public talk page: https://toronto.aitinkerers.org/talks/rsvp_p8BY7gZYuAI

### [Everchanging quest, a Roguelike in which the LLM controls the game](https://paris.aitinkerers.org/talks/rsvp_EuiJ473-gP0)

It is a roguelike in which the layout is passed to the LLM as Markdown. The LLM is then responsible for generating a JSON that contains an objective to fulfill as well as the necessary objects to do so.

- Event context: AI Tinkerers - Paris Inaugural Meetup featuring Mistral AI — 2024-09-03 — Paris
- Public talk page: https://paris.aitinkerers.org/talks/rsvp_EuiJ473-gP0

### [Flux, a new generation of Image Generation Models](https://boston.aitinkerers.org/talks/rsvp_UNy9T277gyY)

The Flux Generation 1 models have taken the image generation community by storm, revolutionizing AI image creation. Text to image has never been easier, and the image to image possibilities are only starting to be explored. How can someone jump into the image generation scene? This talk will cover local tools like ComfyUI and online tools like Replicate. It will also briefly cover LoRAs and what they can do for refined image generation. I don't have a project URL for this yet, but I'm planning to build image generation using Flux into my professional website because some of the APIs are so cheap.

- Event context: August 2024 Meetup at Æthos — 2024-08-26 — Boston
- Public talk page: https://boston.aitinkerers.org/talks/rsvp_UNy9T277gyY

### [Portalverse](https://sf.aitinkerers.org/talks/rsvp_kudAyH_rAIk)

Generate 360 scenes from text in seconds. - website will be deployed later today

- Event context: AI Tinkerers SF X Cerebral Valley - Dec Meetup — 2023-12-04 — San Francisco
- Public talk page: https://sf.aitinkerers.org/talks/rsvp_kudAyH_rAIk

### [Open Source LLM Finetuning OR Hosting Llama / Mistral For $10/month with multi-lora serving](https://berlin.aitinkerers.org/talks/rsvp_GYfKXdF6R9E)

We went through YC's S23 batch and are building a platform for fine-tuning and serving open source LLMs. In general, we can tell a lot about the open source LLM landscape, which ones are currently the best ones available, and why we believe that in the long term, open source LLMs can win against OpenAI (we did not even believe this ourselves a month ago, but some interesting developments took place during the last weeks that changed our mind). To dive deeper into our product and tech, we could talk about two things: 1) how fine-tuning works, and how it improves model performance 2) how we are able to host finetuned models for as little as $10 per month by hosting lora adapters in parallel

- Event context: AI Tinkerers Berlin - November 24 — 2023-11-24 — Berlin
- Public talk page: https://berlin.aitinkerers.org/talks/rsvp_GYfKXdF6R9E

## Related Technologies

- [PyTorch](https://aitinkerers.org/technologies/pytorch) ([Markdown](https://aitinkerers.org/technologies/pytorch.md)) — 273 public demos
- [Python](https://aitinkerers.org/technologies/python) ([Markdown](https://aitinkerers.org/technologies/python.md)) — 662 public demos
- [ComfyUI](https://aitinkerers.org/technologies/comfyui) ([Markdown](https://aitinkerers.org/technologies/comfyui.md)) — 13 public demos
- [OpenAI API](https://aitinkerers.org/technologies/openai-api) ([Markdown](https://aitinkerers.org/technologies/openai-api.md)) — 520 public demos
- [QLoRA](https://aitinkerers.org/technologies/qlora) ([Markdown](https://aitinkerers.org/technologies/qlora.md)) — 4 public demos
- [Qwen](https://aitinkerers.org/technologies/qwen) ([Markdown](https://aitinkerers.org/technologies/qwen.md)) — 18 public demos
- [Agent](https://aitinkerers.org/technologies/agent) ([Markdown](https://aitinkerers.org/technologies/agent.md)) — 12 public demos
- [Anthropic API](https://aitinkerers.org/technologies/anthropic-api) ([Markdown](https://aitinkerers.org/technologies/anthropic-api.md)) — 66 public demos
- [Apertus-8B](https://aitinkerers.org/technologies/apertus-8b) ([Markdown](https://aitinkerers.org/technologies/apertus-8b.md)) — 1 public demo
- [APIs](https://aitinkerers.org/technologies/apis) ([Markdown](https://aitinkerers.org/technologies/apis.md)) — 19 public demos
- [asyncio](https://aitinkerers.org/technologies/asyncio) ([Markdown](https://aitinkerers.org/technologies/asyncio.md)) — 3 public demos
- [BAML](https://aitinkerers.org/technologies/baml) ([Markdown](https://aitinkerers.org/technologies/baml.md)) — 12 public demos
- [base models](https://aitinkerers.org/technologies/base-models) ([Markdown](https://aitinkerers.org/technologies/base-models.md)) — 2 public demos
- [BLIP-2 Q-Former](https://aitinkerers.org/technologies/blip-2-q-former) ([Markdown](https://aitinkerers.org/technologies/blip-2-q-former.md)) — 1 public demo
- [Claude Agent SDK](https://aitinkerers.org/technologies/claude-agent-sdk) ([Markdown](https://aitinkerers.org/technologies/claude-agent-sdk.md)) — 16 public demos
- [ControlNet](https://aitinkerers.org/technologies/controlnet) ([Markdown](https://aitinkerers.org/technologies/controlnet.md)) — 6 public demos
- [CUDA](https://aitinkerers.org/technologies/cuda) ([Markdown](https://aitinkerers.org/technologies/cuda.md)) — 15 public demos
- [Data pipelines](https://aitinkerers.org/technologies/data-pipelines) ([Markdown](https://aitinkerers.org/technologies/data-pipelines.md)) — 1 public demo