# llama Projects at AI Tinkerers

> Canonical HTML: https://aitinkerers.org/technologies/llama
> Markdown URL: https://aitinkerers.org/technologies/llama.md
> Technology record last updated: 2026-02-23T03:58:01Z
> Generated: 2026-09-21T01:34:52Z

Meta's open-weights LLM family optimized for high-performance local deployment and custom fine-tuning across 8B to 405B parameter scales.

Llama 3.1 delivers state-of-the-art performance through a flagship 405B parameter model trained on 15 trillion tokens. It supports a 128k context window: ideal for analyzing massive datasets or long-form documentation. Developers utilize Llama for diverse tasks (multilingual translation, Python code generation, and complex reasoning) while maintaining data sovereignty via local hosting. The ecosystem includes the Llama Stack for agentic workflows and optimized weights for 8B and 70B models, ensuring high throughput on consumer hardware or enterprise clusters.

- Official technology site: https://llama.meta.com/
- Public AI Tinkerers demos and talks: 48
- Result page: 1 of 2

## Recent Public Talks and Demos

### [Create Your Own DeepSeek Moment](https://nyc.aitinkerers.org/talks/rsvp_UswbXaI7O_0)

I built an autonomous loop that tests and proves small models (finetuned in some cases) can replace large ones on real traffic before safely rerouting to them. Live I'll show the decision engine choosing which work to optimize, fine-tuning candidate models, clustering traffic to anonymize data, and if time allows examples of a small model beating a large one on specialized work, an eval that fails and keeps the large model live, and the prove-gate thresholds that make it production-safe.

- Event context: August Demo Day ft Runpod, Veris, Openrouter, — 2026-08-19 — New York City
- Public talk page: https://nyc.aitinkerers.org/talks/rsvp_UswbXaI7O_0

### [LLMs in Your Pocket: Running AI Fully On-Device](https://islamabad-rawalpindi.aitinkerers.org/talks/rsvp_3NT0ZUJ7U5o)

This project demonstrates how to run LLMs entirely on Android devices without relying on the cloud. During the live demo, I'll walk through the complete development workflow, model selection, application architecture, inference pipeline, performance evaluation, and optimization techniques. I'll also showcase the source code, repository structure, logs, and a fully working Android application running an on-device LLM in real time, along with the practical trade-offs and lessons learned while building it.

- Event context: AI Tinkerers Islamabad - August 15, 2026 — 2026-08-15 — Islamabad Rawalpindi
- Public talk page: https://islamabad-rawalpindi.aitinkerers.org/talks/rsvp_3NT0ZUJ7U5o

### [Taskless: I Stopped Writing Tasks and Let My Notes Do It](https://islamabad-rawalpindi.aitinkerers.org/talks/rsvp_R83DVSpQNFA)

Taskless is a local automation that reads my raw, messy Obsidian thought-dumps and turns them into ClickUp tasks on its own, no manual task entry, no project-manager busywork. Live, I'll drop a real note dump into the watched vault and you'll watch n8n pick it up, the LLM classify each line as new / done / progress / ignore, and ClickUp create new tasks and close finished ones in real time from plain prose like "fixed the auth login bug today." I'll show the working n8n workflow, the LLM classification + dedup-by-hash steps, the execution logs as it runs, and the repo.

- Event context: AI Tinkerers Islamabad - July 18, 2026 — 2026-07-18 — Islamabad Rawalpindi
- Public talk page: https://islamabad-rawalpindi.aitinkerers.org/talks/rsvp_R83DVSpQNFA

### [Operating an Agentic Homelab with Agent Guides](https://seattle.aitinkerers.org/talks/rsvp_wvjI8VfiuMM)

I built a multi-node homelab with layered, enterprise-grade security (passwordless auth, secrets management, host hardening) and an OpenTofu plus Ansible control plane for infrastructure and deployment. AI agents operate it instead of me clicking through consoles. In order to facilitate rapid iteration and configuration I created a new open-standard data format called Agent Guides, which are open, Skill-compatible markdown runbooks. Provisioning a new LXC, rotating credentials, or updating firewall settings are repetitive over time, but the details drift and some of the APIs have undocumented behavior, so a single prompt is unreliable and a fixed script breaks. Plain wiki runbooks aren't ideal because they go stale as soon as the environment changes. What I wanted was documented procedures an agent can follow but still improvise when it hits something unexpected. The guides for my homelab are executed primarily by agents, whose harness leaves an audit log to review afterwards, so another agent may update the guide automatically instead of rediscovering the same problem the next time.

- Event context: AI Dev Tools Track - Seattle - July 13 — 2026-07-14 — Seattle
- Public talk page: https://seattle.aitinkerers.org/talks/rsvp_wvjI8VfiuMM

### [Running LLMs on a phone's NPU/GPU](https://nurnberg.aitinkerers.org/talks/rsvp_GoQnkwtTLUg)

PocketPal AI is an open-source mobile app that runs LLMs fully on-device on both iOS and Android. No server, no API key, and works in airplane mode. Live demo: 1. Running models on a phone: chat with a local model, on-device TTS (ONNX), and an agentic loop that builds a simple web page, all on the phone. 2. The dev pipeline (pocketpal-dev-team): a multi-agent Claude Code pipeline that builds the app, then auto-runs e2e tests, takes screenshots and benchmarks across real devices to produce a per-release baseline report. Then I show the stack underneath. Links: https://github.com/a-ghorbani/pocketpal-ai PocketPal AI runs local LLMs on-device. Agentic orchestration framework: https://github.com/a-ghorbani/pocketpal-dev-team Agistry: https://github.com/a-ghorbani/agistry A lightweight, fault-tolerant registry + mailbox for coordinating agent processes. Agistry is a CGO-free Go-based SQLite-backed registry managing token-protected agent data.

- Event context: AI Tinkerers Nürnberg: June Meetup (Community Hub) — 2026-06-24 — Nürnberg
- Public talk page: https://nurnberg.aitinkerers.org/talks/rsvp_GoQnkwtTLUg

### [From Local Agent to Emergent Data Scientist](https://nurnberg.aitinkerers.org/talks/rsvp_T0ZyJGZ5BE0)

A local agent gets a stateful Jupyter kernel and a DuckDB database connection instead of fixed tools, giving it a full Python code execution environment. It has to predict outcomes for head and neck cancer patients from cohort data alone, with the current patient's outcome hidden. What I didn't design: the agent trained its own ML models and weighted their outputs based on the patient's individual features, unprompted. Live: I'll show agent traces and behavior in Logfire, walk through the system design, and explain the sandboxed code execution setup.

- Event context: AI Tinkerers Nürnberg: June Meetup (Community Hub) — 2026-06-24 — Nürnberg
- Public talk page: https://nurnberg.aitinkerers.org/talks/rsvp_T0ZyJGZ5BE0

### [What you can do with a used GPU](https://nurnberg.aitinkerers.org/talks/rsvp_YlYa6jJt-us)

A private Telegram-based personal knowledge assistant that lets me save notes from my phone, store and retrieve them with ChromaDB on a VPS accessible anytime, and ask questions answered by a local llama.cpp model running on my home GPU over a private network i setup. Will show probably everything, or as much as i can in the time i have, but demo's are fun and I want to show how the information propagates through what parts of the code to what parts of my telegram UX and use it on stage with suggestions from the crowd for notes. and maybe also show how we can change the prompt handling this assistant on the fly without touching the code using the agenta prompt management platform. maybe also show how i setup lamacpp for maximum inference time even for low vram machines with big beefy models.

- Event context: AI Tinkerers Nürnberg: June Meetup (Community Hub) — 2026-06-24 — Nürnberg
- Public talk page: https://nurnberg.aitinkerers.org/talks/rsvp_YlYa6jJt-us

### [Prism: Privacy-First Multi-Model Orchestration](https://dubai.aitinkerers.org/talks/rsvp_gu_mOBH3NEM)

Prism is a privacy-aware AI orchestration layer that decomposes any query into parallel sub-tasks, routes each to the right model (local Qwen3, Groq, or Gemini) based on sensitivity, executes them concurrently via a dependency DAG, then synthesizes a single coherent response — all in one WebSocket-driven interface. Live demo will show: a real query decomposing into 3 parallel tasks, a DAG rendering the execution graph in real-time, the privacy gate redacting secrets before they leave the machine, and a local Qwen3.6-35B running on a laptop GPU via llama.cpp with MoE CPU offload.

- Event context: AI Tinkerers Dubai - June Demo Day — 2026-06-13 — Dubai
- Public talk page: https://dubai.aitinkerers.org/talks/rsvp_gu_mOBH3NEM

### [Sarathy & me!](https://ottawa.aitinkerers.org/talks/rsvp_bcfMBtYyxgc)

A openclaw style personal assistant forked from nanobot, inspired from Hermes and openclaw

- Event context: AI Tinkerers Ottawa Meetup — April 25th, 2026 — 2026-04-25 — Ottawa
- Public talk page: https://ottawa.aitinkerers.org/talks/rsvp_bcfMBtYyxgc

### [Seams Showing: Deconstructionist Approach to AI Engineering](https://dc.aitinkerers.org/talks/rsvp_M_8O9nWG6RU)

AO is a governed AI pipeline for MTG rules adjudication. Instead of a chat loop, it's a layered system where each stage — input, retrieval, semantic gating, and response — has an explicit contract and a single owner. The interface is built around a single animated spine element that reflects system state rather than a chat UI. Inference runs fully on-device.

- Event context: AI Tinkerers x Softengi - DC Metro Meetup - April 9th, 2026 — 2026-04-09 — DC
- Public talk page: https://dc.aitinkerers.org/talks/rsvp_M_8O9nWG6RU

### [AI-to-USD: An LLM Agent That Self-Corrects Industrial Scene Generation for Isaac Sim](https://nyc.aitinkerers.org/talks/rsvp_Y4sGAt1oy-A)

I built an end-to-end pipeline that turns natural language descriptions into validated USD industrial scenes and synthetic training data for computer vision. The pipeline has two stages: Stage 1: Spec-to-Sim Copilot: You describe a battery module assembly task ("assemble a 2x3 grid of LG E63 cells with a UR10e robot"). An LLM (Gemini 2.0 Flash with structured output) generates a Pydantic-validated ModuleTask JSON. Then, 5 industrial safety rules are checked against the spec: thermal spacing between cells, robot arm reachability, cell rotation alignment for busbar welding, module tray bounds, and cell count limits. If validation fails, the system feeds structured error reports back to the LLM for self-correction (up to 3 repair attempts). This is the interesting part: pure LLM generation fails silently for spatial tasks — cells overlap by 5-10mm, coordinates fall outside robot reach — so the validation loop catches what the LLM can't reason about. Stage 2: Industrial SDG Lab: The validated scene gets domain randomization across 4 axes (lighting intensity, material color shift, camera pose, object placement jitter) to produce variant USDA scenes with COCO-format annotations — ready for training object detection models. An Omniverse-style 3D viewport built with Three.js lets you interactively inspect the scene with PBR materials, bloom post-processing, and a property inspector panel. Live demo: I'll type a prompt, show the LLM generating a spec, trigger a validation failure, watch the AI self-correct, then generate randomized variants with the 3D viewport.

- Event context: March Demo Day, hosted by Flowglad — 2026-03-18 — New York City
- Public talk page: https://nyc.aitinkerers.org/talks/rsvp_Y4sGAt1oy-A

### [QuAI: The Vibe-Coded AI Writing Assistant I Use Every Day](https://montreal.aitinkerers.org/talks/rsvp_c6ieg8zqrj4)

I'll be presenting how I used vibe-coding to solve a personal challenge and showcasing my solution: QuAI, a desktop app I use daily for writing on my Mac. I will also discuss the tech stack and tools I utilized. Since it was vibe-coded, I don't even know all the details; however, because people are now asking for access, I'm considering moving toward agentic engineering, where I will actually deep-dive into the code and structure.

- Event context: AI Tinkerers Montreal - February 2026 Meetup — 2026-02-24 — Montreal
- Public talk page: https://montreal.aitinkerers.org/talks/rsvp_c6ieg8zqrj4

### [Local hosting - sometimes joy can come in small packages](https://hong-kong.aitinkerers.org/talks/rsvp_cW9hxrT3uxI)

Often people consider AI inferencing as something that can only be done on cloud. And if it's to be done locally, it needs to be done in a large computer case - culminating in a hot and loud system that's sure to keep everyone at home in an annoyed state. I'll be presenting on how local AI inferencing doesn't need to be full of compromises, but also some of the different approaches to building out a system for local hosting. As somebody with a passion for building small form-factor PCs (SFFPCs), I've had a lot of experience building small and space-efficient desktops. Unlike building a SFFPC that can run games quietly and with good temperatures, I'll be showing how an AI-capable SFFPC is actually a simpler affair than building a gaming system. I'll also go through the different ways to host locally, with a particular focus on Windows applications, as well as Ubuntu via WSL. To meet the requirement of applied AI in finance, I will also be showing a new version of the Counsel Intelligence model that runs locally on my desktop.

- Event context: AI Tinkerers Hong Kong & GBA: Applied AI in Finance Demos — 2026-01-20 — Hong Kong
- Public talk page: https://hong-kong.aitinkerers.org/talks/rsvp_cW9hxrT3uxI

### [NeuroGraph](https://paris.aitinkerers.org/talks/rsvp_BSibVS972M0)

A Graph Builder based on a file or URL

- Event context: Turn Your Knowledge into an API for LLMs - Meetup — 2025-12-09 — Paris
- Public talk page: https://paris.aitinkerers.org/talks/rsvp_BSibVS972M0

### [Document Analytics with Open weight models for Sovereign Data](https://berlin.aitinkerers.org/talks/rsvp_5W86mvFdiH0)

get started with Document analytics in < 100 lines of code. PDF to Image -> VLM for OCR with Image -> Structured JSON Response -> Text to SQL with Tool Call -> Uses Qwen-3-VL-4B-Instruct

- Event context: AI Tinkerers Berlin Meetup - November 12th, 2025 — 2025-11-12 — Berlin
- Public talk page: https://berlin.aitinkerers.org/talks/rsvp_5W86mvFdiH0

### [LLMs for retrieval and recommendation](https://toronto.aitinkerers.org/talks/rsvp_fnoZbMMW_ao)

I will show how to use vLLM and an openweight model to make a simple recommendation engine and use guided decoding to limit the output of the llm to the allowed items only. No finetuning needed and it will work on google colab so basically no hardware needed either. the code i shared will be a bit more, that is just a draft.

- Event context: AI Tinkerers Toronto - October 2025 Meetup at CIBC | Simplii — 2025-10-30 — Toronto
- Public talk page: https://toronto.aitinkerers.org/talks/rsvp_fnoZbMMW_ao

### [Jailbreaking Small Language Models](https://seattle.aitinkerers.org/talks/rsvp_YEW056rGuLg)

SLMs are popular efficient alternatives to burning tokens. However are they equally safe to use?

- Event context: Trust & Security in AI: Demo Night w/ Auth0 — 2025-10-30 — Seattle
- Public talk page: https://seattle.aitinkerers.org/talks/rsvp_YEW056rGuLg

### [EVA - Enhanced Video Archive](https://raleigh.aitinkerers.org/talks/rsvp_9ld1Kd98tv4)

EVA (Enhanced Video Archive) turns sprawling legacy video collections into a searchable knowledge base by indexing transcripts, descriptions, and keyframes, then using RAG to answer text or screenshot queries. Ask for a topic and EVA returns a report with linked screenshots and exact timestamps—so trainers and learners can jump straight to the moments that matter instead of scrubbing through hours of footage.

- Event context: AI Tinkerers - Raleigh Inaugural Meetup (September 2025) — 2025-09-30 — Raleigh
- Public talk page: https://raleigh.aitinkerers.org/talks/rsvp_9ld1Kd98tv4

### [SLM for MacOs](https://poland.aitinkerers.org/talks/rsvp_kDFlECrDcVM)

Challenges and opportunities of using SLM on MacOs platform

- Event context: AI Tinkerers Poland #5 - Meetup in Warsaw (September) — 2025-09-18 — Poland
- Public talk page: https://poland.aitinkerers.org/talks/rsvp_kDFlECrDcVM

### [Parsing complex pdfs using vision models](https://dubai.aitinkerers.org/talks/rsvp_DTChla5ieP8)

My experience converting complex PDFs (containing tables and math formulas) using lama4 maverick and Qwen 2.5 VI. I want to show the importance of image resolution in having good text/formulas/table extraction. Hopefully this will save some time for those trying to do the same thing

- Event context: AI Tinkerers Dubai Meetup – June 2025 Demo Day — 2025-06-28 — Dubai
- Public talk page: https://dubai.aitinkerers.org/talks/rsvp_DTChla5ieP8

### [Put your idle devices to work and become your own AI cloud](https://liverpool.aitinkerers.org/talks/rsvp_FBkKgRje2gE)

I am the founder of Kalavai, a tool that turns any device into a scalable platform for GenAI. It helps developers aggregate compute from any source (cloud, on prem, laptops) in a unified layer, and manages one-click distributed deployment of AI models. What I'd love to present is our new LLM queuing system, which is an asynchronous batch processing queue that helps developers optimise their workloads at scale, much faster and cost effective than real time inference. Behind the scenes, the queue handles batch requests, whilst workers (any computer) pick inference jobs and run them locally. Workers do just-in-time model deployment, which means there is no model idle time (costly!). And because inference jobs are batched, we can optimise model throughput by configuring batch size. During the presentation, I plan to show how easy it is to run a batch evaluation of multiple models on a test dataset on my computer. This will demonstrate: 1) the power of personal computing devices 2) how queuing maximises hardware utilisation 3) how easy it is to auto deploy models

- Event context: AI Tinkerers - Liverpool Inaugural Meetup (June) — 2025-06-26 — Liverpool
- Public talk page: https://liverpool.aitinkerers.org/talks/rsvp_FBkKgRje2gE

### [Personal assistant using low code AI tools](https://nairobi.aitinkerers.org/talks/rsvp_fEY6XpQMjiE)

I would like to showcase to the audience how to get started with AI tools using low code tools such as n8n . In this demo - I will show how I created a personal assistant using n8n and using Llama parsing AI API that runs on top of Llama. My personal assistant can get me my calendar events for the week and I can take a picture and it explains to me what the picture means right on my WhatsApp.

- Event context: AI Tinkerers - Nairobi #2 - June 2025 — 2025-06-11 — Nairobi
- Public talk page: https://nairobi.aitinkerers.org/talks/rsvp_fEY6XpQMjiE

### [Hyper personalized guidance to improve health.](https://nyc.aitinkerers.org/talks/rsvp_FL1OhKKB1PA)

80% of healthcare diseases are preventable and reversible with simple, early interventions Live150 empowers individuals seeking better health, and experts scaling consumer health services to promote wellness and longevity. We offer frictionless, AI-driven, hyper-personalized guidance on nutrition, activity, mindfulness, and sleep, empowering users with continuous, clear, and actionable steps for optimal health. Live150 goes beyond tracking, offering effortless data capture and personalized guidance, like Google Maps for your health—keeping you on course to wellness. I reversed my own health issues around high cholesterol, sugar and BP, through personalized lifestyle changes—and now I’m on a mission to help others do the same.

- Event context: Demos and Dim Sum with Deel and Apollo — 2025-06-03 — New York City
- Public talk page: https://nyc.aitinkerers.org/talks/rsvp_FL1OhKKB1PA

### [Artecon - A hotspot for AI](https://seattle.aitinkerers.org/talks/rsvp_LHdsMBc1mpg)

An inference engine running a variety of machine learning workloads on commodity hardware, making prototyping easy, private and cheap. The project places a special focus on ease-of-use. The goal is to have an app that allows even non-technical users to use common models locally or turn their PC/laptop into a hotspot for others to use. (i) The app includes an LLM that uses tools to respond to queries. Each tool itself might contain models (object detector, vision-language models, etc.). (ii) Users can run the app one PC/laptop and easily allow friends to share this through a discord/telegram/signal group chat.

- Event context: Friday - AI Tinkerers Seattle - May Meetup — 2025-05-31 — Seattle
- Public talk page: https://seattle.aitinkerers.org/talks/rsvp_LHdsMBc1mpg

## Related Technologies

- [Python](https://aitinkerers.org/technologies/python) ([Markdown](https://aitinkerers.org/technologies/python.md)) — 662 public demos
- [Mistral](https://aitinkerers.org/technologies/mistral) ([Markdown](https://aitinkerers.org/technologies/mistral.md)) — 24 public demos
- [OpenAI API](https://aitinkerers.org/technologies/openai-api) ([Markdown](https://aitinkerers.org/technologies/openai-api.md)) — 520 public demos
- [vLLM](https://aitinkerers.org/technologies/vllm) ([Markdown](https://aitinkerers.org/technologies/vllm.md)) — 33 public demos
- [Qwen](https://aitinkerers.org/technologies/qwen) ([Markdown](https://aitinkerers.org/technologies/qwen.md)) — 18 public demos
- [Docker](https://aitinkerers.org/technologies/docker) ([Markdown](https://aitinkerers.org/technologies/docker.md)) — 147 public demos
- [Gemini](https://aitinkerers.org/technologies/gemini) ([Markdown](https://aitinkerers.org/technologies/gemini.md)) — 188 public demos
- [GGUF](https://aitinkerers.org/technologies/gguf) ([Markdown](https://aitinkerers.org/technologies/gguf.md)) — 5 public demos
- [LangChain](https://aitinkerers.org/technologies/langchain) ([Markdown](https://aitinkerers.org/technologies/langchain.md)) — 445 public demos
- [ONNX](https://aitinkerers.org/technologies/onnx) ([Markdown](https://aitinkerers.org/technologies/onnx.md)) — 83 public demos
- [OpenAI](https://aitinkerers.org/technologies/openai) ([Markdown](https://aitinkerers.org/technologies/openai.md)) — 112 public demos
- [React](https://aitinkerers.org/technologies/react) ([Markdown](https://aitinkerers.org/technologies/react.md)) — 219 public demos
- [Claude](https://aitinkerers.org/technologies/claude) ([Markdown](https://aitinkerers.org/technologies/claude.md)) — 173 public demos
- [FastAPI](https://aitinkerers.org/technologies/fastapi) ([Markdown](https://aitinkerers.org/technologies/fastapi.md)) — 181 public demos
- [Flux](https://aitinkerers.org/technologies/flux) ([Markdown](https://aitinkerers.org/technologies/flux.md)) — 8 public demos
- [GPT-4](https://aitinkerers.org/technologies/gpt-4) ([Markdown](https://aitinkerers.org/technologies/gpt-4.md)) — 529 public demos
- [GPT-4o](https://aitinkerers.org/technologies/gpt-4o) ([Markdown](https://aitinkerers.org/technologies/gpt-4o.md)) — 57 public demos
- [Gradio](https://aitinkerers.org/technologies/gradio) ([Markdown](https://aitinkerers.org/technologies/gradio.md)) — 9 public demos

## More Results

- Next: https://aitinkerers.org/technologies/llama.md?page=2