# QLoRA Projects at AI Tinkerers

> Canonical HTML: https://aitinkerers.org/technologies/qlora
> Markdown URL: https://aitinkerers.org/technologies/qlora.md
> Technology record last updated: 2026-02-23T16:21:27Z
> Generated: 2026-09-21T02:47:06Z

QLoRA (Quantized Low-Rank Adaptation) is an efficient finetuning approach: it uses 4-bit quantization and Low-Rank Adapters (LoRA) to drastically reduce Large Language Model (LLM) memory usage without sacrificing performance.

QLoRA is a game-changer for LLM fine-tuning, enabling operations previously deemed infeasible. The core mechanism backpropagates gradients through a frozen, 4-bit quantized pretrained model directly into smaller, 16-bit LoRA adapters. This technique cuts memory requirements: you can fine-tune a 65B parameter model on a single 48GB GPU, preserving full 16-bit performance. Key innovations include 4-bit NormalFloat (NF4) quantization, Double Quantization, and Paged Optimizers for managing memory spikes. The resulting Guanaco model family, for example, achieved 99.3% of ChatGPT's performance on the Vicuna benchmark, demonstrating state-of-the-art results with consumer-grade hardware access.

- Official technology site: https://arxiv.org/abs/2305.14314
- Public AI Tinkerers demos and talks: 4
- Result page: 1 of 1

## Recent Public Talks and Demos

### [Speak mk1: A multimodal mamba-attention hybrid model for speech therapy](https://dubai.aitinkerers.org/talks/rsvp_f9s0FLsNTT0)

I am currently building a mamba-attention hybrid encoder and decoder from scratch for speech therapy, not fine-tuning, not wrapping an API, but training every component end to end on my RTX 4060 laptop. The system has three components I am building in parallel: The first is a custom Mamba SSM-based audio encoder trained on LibriSpeech with multi-task phonological heads for voicing, manner, place of articulation, and correctness detection. The encoder uses a BLIP-2-style Q-Former to bridge audio representations into the language model. The second is a video pipeline using MediaPipe FaceLandmarker to isolate and analyze oral region motion frame by frame, extracting articulatory features like tongue tip position, mouth opening geometry, and lip protrusion in real time from a standard webcam. The third component is SpeakMK1LLM, which serves as the project’s core reasoning engine. To balance cutting-edge architectural research with deployment stability for the current phase, we have developed two iterations of this model: The Hybrid Prototype: A custom 70M parameter Mamba-attention hybrid model designed for high-efficiency sequence modeling. It was trained on a rigorous four-stage curriculum: general pretraining on Tinystories, domain adaptation on CHILDES, clinical knowledge injection from pubmed central, and final instruction tuning. The Deployment Model: A fine-tuned Gemma 3n E4B, which leverages the same four-stage curriculum to adapt its large-scale reasoning capabilities to the specific nuances of Speech-Language Pathology (SLP). The entire stack requires only a webcam and a microphone, which is the point — making clinical-quality articulation feedback accessible without specialized hardware or a therapist present in the room. For the demo I would show the live pipeline taking a child's speech, flagging a specific phoneme error, and generating a graded corrective prompt in real time, alongside the clinician dashboard that logs session data for remote review.

- Event context: AI Tinkerers Dubai - May Demo Day — 2026-05-23 — Dubai
- Public talk page: https://dubai.aitinkerers.org/talks/rsvp_f9s0FLsNTT0

### [Fine-Tuning Models for Content Moderation with Apple’s MLX Framework](https://orange-county.aitinkerers.org/talks/rsvp_S0EeHEPZUv4)

This talk is a hands-on walkthrough of how to fine-tune large language models—and multi-modal models—locally on Apple-Silicon Macs using Apple’s open-source MLX framework. We’ll: 1. Give a quick MLX overview and why it’s optimized for the M-series GPU/ANE. 2. Explain LoRA / QLoRA and why adapter-based fine-tuning is memory-efficient. 3. Dissect the vision-encoder → adapter → LLM pipeline (SigLIP + Phi-1.5). 4. Show dataset prep in JSONL, CLI commands (mlx_lm.lora, mlx_lm.fuse) and YAML options. 5. Live-interpret training logs, validation curves, and evaluation metrics. 6. End with best-practice checklists for scaling from small experiments to full production runs. Attendees will leave able to replicate the full workflow—dataset → training → evaluation → deployment—entirely on their MacBooks.

- Event context: AI Tinkerers - Orange County Meetup- Wednesday June 4th 2025 — 2025-06-05 — Orange County
- Public talk page: https://orange-county.aitinkerers.org/talks/rsvp_S0EeHEPZUv4

### [Can Identity be fine-tuned with QLoRA for an AI Vtuber?](https://quito.aitinkerers.org/talks/rsvp_ihNPM41KEw4)

Can Identity be fine-tuned with QLoRA for an AI Vtuber? In this technical demonstration, I'll walk you through the step-by-step creation a real-time interactive VTuber powered by a base LLM (LLaMA 3.2) 3B*, fine-tuned with QLoRA and the Unsloth framework. You'll see firsthand how we went beyond typical "instruction-following" models to craft a uniquely creative and reflexive virtual personality. I'll share: How we efficiently applied LoRA layers for lightweight, GPU-friendly fine-tuning. Deep insights into dataset preparation, custom prompt engineering, and careful handling of conversational structure to create a convincingly human-like personality. Technical challenges overcome, including managing VRAM constraints (on an RTX 3060) through quantization, CPU offloading, and careful gradient management.

- Event context: AI Tinkerers - Quito Primer Meetup (Abril) — 2025-04-24 — Quito
- Public talk page: https://quito.aitinkerers.org/talks/rsvp_ihNPM41KEw4

### [Fine-tuning LLMs to generate dad jokes](https://berlin.aitinkerers.org/talks/rsvp_C-oPmXiOg7M)

If you've ever tried to ask ChatGPT to make a dad joke, you most probably know how flat and boring such jokes are. To make the world a better place we trained an experimental llama2/llama3/mistral LLM to generate a dad joke punchline to any prompt you give based on a vast collection of training data we found on Reddit.

- Event context: Ollama & Friends coming to AI Tinkerers Berlin — 2024-07-18 — Berlin
- Public talk page: https://berlin.aitinkerers.org/talks/rsvp_C-oPmXiOg7M

## Related Technologies

- [Llama 3](https://aitinkerers.org/technologies/llama-3) ([Markdown](https://aitinkerers.org/technologies/llama-3.md)) — 38 public demos
- [LoRA](https://aitinkerers.org/technologies/lora) ([Markdown](https://aitinkerers.org/technologies/lora.md)) — 17 public demos
- [Python](https://aitinkerers.org/technologies/python) ([Markdown](https://aitinkerers.org/technologies/python.md)) — 662 public demos
- [PyTorch](https://aitinkerers.org/technologies/pytorch) ([Markdown](https://aitinkerers.org/technologies/pytorch.md)) — 273 public demos
- [Axolotl](https://aitinkerers.org/technologies/axolotl) ([Markdown](https://aitinkerers.org/technologies/axolotl.md)) — 1 public demo
- [BLIP-2 Q-Former](https://aitinkerers.org/technologies/blip-2-q-former) ([Markdown](https://aitinkerers.org/technologies/blip-2-q-former.md)) — 1 public demo
- [Facial Landmark Detection](https://aitinkerers.org/technologies/facial-landmark-detection) ([Markdown](https://aitinkerers.org/technologies/facial-landmark-detection.md)) — 1 public demo
- [Gemma 3n E4B](https://aitinkerers.org/technologies/gemma-3n-e4b) ([Markdown](https://aitinkerers.org/technologies/gemma-3n-e4b.md)) — 1 public demo
- [Llama-2](https://aitinkerers.org/technologies/llama-2) ([Markdown](https://aitinkerers.org/technologies/llama-2.md)) — 227 public demos
- [Mamba-2](https://aitinkerers.org/technologies/mamba-2) ([Markdown](https://aitinkerers.org/technologies/mamba-2.md)) — 1 public demo
- [MediaPipe FaceLandmarker](https://aitinkerers.org/technologies/mediapipe-facelandmarker) ([Markdown](https://aitinkerers.org/technologies/mediapipe-facelandmarker.md)) — 1 public demo
- [Mistral](https://aitinkerers.org/technologies/mistral) ([Markdown](https://aitinkerers.org/technologies/mistral.md)) — 24 public demos
- [MLX-LM](https://aitinkerers.org/technologies/mlx-lm) ([Markdown](https://aitinkerers.org/technologies/mlx-lm.md)) — 5 public demos
- [Modal](https://aitinkerers.org/technologies/modal) ([Markdown](https://aitinkerers.org/technologies/modal.md)) — 14 public demos
- [Qwen](https://aitinkerers.org/technologies/qwen) ([Markdown](https://aitinkerers.org/technologies/qwen.md)) — 18 public demos
- [Unsloth Framework](https://aitinkerers.org/technologies/unsloth-framework) ([Markdown](https://aitinkerers.org/technologies/unsloth-framework.md)) — 1 public demo
- [wav2vec 2](https://aitinkerers.org/technologies/wav2vec-2) ([Markdown](https://aitinkerers.org/technologies/wav2vec-2.md)) — 2 public demos