# CUDA Projects at AI Tinkerers

> Canonical HTML: https://aitinkerers.org/technologies/cuda
> Markdown URL: https://aitinkerers.org/technologies/cuda.md
> Technology record last updated: 2026-02-23T10:56:56Z
> Generated: 2026-09-22T01:43:55Z

CUDA is NVIDIA's parallel computing platform: it unlocks the massive core count of GPUs (Graphics Processing Units) for accelerated general-purpose processing.

CUDA (Compute Unified Device Architecture) is NVIDIA's proprietary parallel computing platform and API, officially released in 2007. It transforms the GPU from a graphics processor into a high-performance compute engine, allowing developers to offload compute-intensive tasks from the CPU to the GPU's thousands of cores for massive parallelism . The CUDA Toolkit provides the necessary environment: a C/C++ compiler, libraries (like cuBLAS, cuFFT), and debugging tools . This framework is the foundation for modern High-Performance Computing (HPC), delivering significant speed-ups (e.g., 30x-100x for certain workloads) across critical fields like AI (deep learning model training), scientific simulations, and computational finance .

- Official technology site: https://developer.nvidia.com/cuda-zone
- Public AI Tinkerers demos and talks: 15
- Result page: 1 of 1

## Recent Public Talks and Demos

### [How does GPU memory allocation strategy affect graph analytics performance?](https://seattle.aitinkerers.org/talks/rsvp_Iz_LtazuvjU)

I built a CUDA-based GPU Memory Allocation Benchmark Framework that studies how different memory strategies impact graph analytics workloads.The motivation came from the fact that many real-world applications, like social networks, fraud detection, recommendation systems, and scientific computing, rely on large graph structures where memory access patterns are irregular and memory behavior can become a performance bottleneck.To investigate this, I implemented a PageRank graph workload and created a controlled benchmark where the graph, algorithm, GPU kernel, and number of iterations stayed exactly the same. The only variable I changed was the GPU memory allocation strategy.

- Event context: Women in AI - Builders Track: Seattle Meetup — 2026-08-19 — Seattle
- Public talk page: https://seattle.aitinkerers.org/talks/rsvp_Iz_LtazuvjU

### [On-premise AI solution for Cloud PBX provider](https://valencia.aitinkerers.org/talks/rsvp_zguJ0GbdGVs)

On-premise AI transcription. I rebuild faster-whisper lib to make it more efficient for dual-channel transcription. I've built GPU-servers infrastructure for CloudPBX providers with transcirption and analysys.

- Event context: AI Tinkerers Valencia April Meetup — 2026-04-21 — Valencia
- Public talk page: https://valencia.aitinkerers.org/talks/rsvp_zguJ0GbdGVs

### [Vibe Coding Historical Nuclear Tests](https://la.aitinkerers.org/talks/rsvp_JkUiZuZtKkE)

FSRM (Full Service Reservoir Model) demonstrates the power of AI-assisted development to create sophisticated scientific software for nuclear explosion monitoring. Using "vibecoding"—iterative collaboration with large language models—a comprehensive multi-physics simulator was developed that accurately reproduces historical nuclear test signatures for both underground and atmospheric detonations. The code implements Mueller-Murphy seismic source models for underground tests, capturing cavity formation, spall, chimney collapse, and seismic wave generation essential for treaty verification. Atmospheric detonation models include Brode fireball evolution, Sedov-Taylor blast wave propagation, and electromagnetic pulse (EMP) effects. Built on PETSc for parallel computing, the simulator handles complex coupling between elastodynamics, plasticity, fracture mechanics, and radiation transport. This "vibecoded" approach enabled rapid development of capabilities traditionally requiring years of specialized coding, producing a tool that validates against SCEC earthquake benchmarks and generates synthetic seismograms comparable to real IMS station data. The resulting open-source code provides nuclear monitoring agencies with a flexible platform for testing detection algorithms, analyzing historical events, and training analysts on explosion phenomenology. The project illustrates how AI-assisted development can democratize access to sophisticated geophysical simulation tools previously limited to national laboratories.

- Event context: AI Tinkerers LA: March Builder Meetup & Live Demos — 2026-03-20 — Los Angeles
- Public talk page: https://la.aitinkerers.org/talks/rsvp_JkUiZuZtKkE

### [Personal AI Supercomputers: From Cloud Dependency to Local AI](https://paris.aitinkerers.org/talks/rsvp_k6HVVqqlz3s)

AI development is entering a new era where developers no longer need massive cloud clusters to build and run advanced models. A new class of AI-native personal supercomputers, powered by NVIDIA’s Grace-Blackwell architecture, is bringing datacenter-grade AI capabilities directly to the developer desk. In this talk, we will explore how systems like NVIDIA DGX Spark and Lenovo ThinkStation PGX are reshaping the way AI engineers prototype, train, and deploy models locally — from large language models to multimodal and agentic AI systems. We will explain the architecture behind the GB10 Grace-Blackwell superchip, unified memory systems, and the NVIDIA AI software stack that makes these platforms powerful tools for experimentation and enterprise AI development. This session will answer key questions such as: Why are AI personal supercomputers emerging as a new category of computing? What problems do developers face today with cloud-only AI development? How do DGX Spark and Lenovo ThinkStation PGX enable developers to run models up to hundreds of billions of parameters locally? How does unified memory and low-precision computing (FP4/FP8) accelerate modern AI workloads? What role does the NVIDIA AI ecosystem (NeMo, NIM, Blueprints, CUDA libraries) play in building AI agents and applications? How does the AI development workflow evolve from prototyping to deployment across personal, enterprise, and cloud systems? We will also demonstrate how these systems enable developers to move seamlessly from experimentation to production-scale AI while maintaining performance, security, and cost efficiency.

- Event context: High-Performance Local AI Development: Kick-off ThinkStation PGX — 2026-03-17 — Paris
- Public talk page: https://paris.aitinkerers.org/talks/rsvp_k6HVVqqlz3s

### [Benchmarking Small Language Models Where It Actually Matters](https://paris.aitinkerers.org/talks/rsvp_ow7csxTS5J0)

Most SLM benchmarks answer the wrong question. They tell you how a model scores — not whether it works. This platform is designed for teams who care about real execution, not paper metrics. It lets you benchmark Small Language Models on Python and Polars code generation, under strictly controlled hardware conditions, with full visibility on performance, cost, and failure modes. Participants connect to the platform through a web interface and run benchmarks on large datasets and realistic workloads. The backend runs inside a Docker environment and can execute jobs directly on GPUs, whether locally or on dedicated infrastructure. Each run is configurable: quantization, decoding parameters, and runtime settings are part of the experiment, not hidden defaults. The focus on Polars is intentional. Most language models are very good at generating code for older, widely used libraries like pandas or NumPy. These libraries have been present in training data for years and appear in millions of examples online. Polars is different. It is a newer, high-performance data processing library designed for large-scale workloads, built around vectorized execution, query planning, and expression-based transformations. While it offers major performance advantages for large datasets, its programming model is significantly different from traditional Python data tools. As a result, many models struggle with it. They may generate code that looks correct but fails to run, produces incorrect results, or uses inefficient patterns that defeat the performance benefits of the library. This makes Polars an ideal stress test for evaluating whether a model truly understands modern data-processing workflows. Beyond model metrics (tokens/sec, VRAM usage, GPU utilization), the platform evaluates the generated code itself: Does it run? Does it produce the correct result? Is it efficient, or just “technically correct but slow”? To make progress measurable and engaging, the platform also supports a hackathon-style workflow: Real-time leaderboards Full attempt history per team and per benchmark Analytics dashboards highlighting where models systematically fail The goal is simple: give teams a new language to talk about model quality — not “good vs bad,” but correct, efficient, reliable, and production-ready. If you don’t measure this, you might end up shipping models that look good in demos… and quietly break in real workloads.

- Event context: High-Performance Local AI Development: Kick-off ThinkStation PGX — 2026-03-17 — Paris
- Public talk page: https://paris.aitinkerers.org/talks/rsvp_ow7csxTS5J0

### [Continuously Learning AI in Regulated Environments: Versioned Models, Secure Execution, and Controlled Adaptation](https://milan.aitinkerers.org/talks/rsvp_cCBE4KuVMmU)

Regulated industries face a fundamental tension: AI systems need to adapt and improve, yet compliance requires reproducibility, auditability, and strict execution controls. Hyprstream addresses this challenge by treating models as versioned, Git-native artifacts. Base models and fine-tuned adapters are stored as repositories, enabling branching, commit history, and reproducible rollbacks. Every model state becomes traceable. Every adaptation becomes auditable. Within a unified runtime, Hyprstream supports both inference serving and LoRA-based fine-tuning, allowing systems to learn from domain-specific data while remaining inside controlled infrastructure. Model branches can be deployed as distinct OpenAI-compatible endpoints, enabling parallel validation, staged rollouts, and controlled promotion to production. Security is layered by design: Transport encryption and signed RPC envelopes Policy-based authorization Isolated execution via microVM-backed workers Local-first deployment without mandatory SaaS dependencies The result is an architecture that supports continuously learning applications while meeting the demands of compliance, traceability, and operational control. This talk explores how AI systems can evolve without sacrificing governance — and how regulated environments can adopt adaptive AI without relinquishing oversight.

- Event context: AI Tinkerers Milan - February 24, 2026 - Agentic Orchestration in Financial Services: Architectures & Demos — 2026-02-24 — Milan
- Public talk page: https://milan.aitinkerers.org/talks/rsvp_cCBE4KuVMmU

### [On the Impact of Black-box Deployment Strategies for Edge AI on Latency and Model Performance](https://toronto.aitinkerers.org/talks/rsvp_T1O04EKQQPE)

This presentation will explore the findings from our paper “On the Impact of Black-box Deployment Strategies for Edge AI on Latency and Model Performance.” The study empirically evaluates how different Edge AI deployment strategies—specifically combinations of Partitioning, Quantization, and Early Exit operators—affect inference latency and accuracy across Mobile, Edge, and Cloud environments. Using ONNX-based models deployed in a containerized environment, the research examines both single- and multi-tier configurations (e.g., Mobile-Edge, Edge-Cloud) under varying network bandwidths. Results highlight that hybrid strategies, particularly Quantization + Early Exit on Edge, offer optimal latency-accuracy trade-offs for many real-world conditions, while Quantization alone is best when accuracy preservation is critical. These findings provide actionable insights for MLOps engineers seeking efficient, privacy-aware deployment strategies in heterogeneous Edge AI systems.

- Event context: AI Tinkerers Toronto - November 2025 Meetup at Shopify! — 2025-11-10 — Toronto
- Public talk page: https://toronto.aitinkerers.org/talks/rsvp_T1O04EKQQPE

### [Slashing GPU Kernel Launch Overhead in LLM Inference](https://berlin.aitinkerers.org/talks/rsvp_ghQq_D2zfys)

LLM inference often involves millions of kernel launches. The token-by-token nature of decode workloads bombards the GPU with smaller, sequential operations, creating a hidden bottleneck in driver overhead, not compute. We'll demonstrate a multi-process Rust application that decouples inference logic from GPU execution using a lock-free shared memory queue. This architecture enables intelligent, on-the-fly batching of identical operations into a single, efficient cuBLASLt call. The results are significant: we'll show a 90%+ reduction in kernel launches and a 22% speedup on a realistic FP16 decode workload. We'll also explore why this same technique results in a slight slowdown for compute-bound prefill workloads, providing a nuanced, first-principles look at a core optimization used by all major inference engines. Update: GitHub repo added

- Event context: AI Tinkerers Meetup August 13 — 2025-08-13 — Berlin
- Public talk page: https://berlin.aitinkerers.org/talks/rsvp_ghQq_D2zfys

### [Beyond Text: Building a fast Visual Search Engine](https://berlin.aitinkerers.org/talks/rsvp_7LBNQm7JTWM)

This demo will dive deep into the architecture and implementation of Youzu Lens, our flagship zero-shot visual search engine. We will explore how we leverage vector models to generate rich, 1280-dimensional image embeddings, enabling precise product recognition and similarity matching. The session will cover the pipeline, from multi-tenant product ingestion and GPU-accelerated inference (PyTorch/CUDA) to high-performance vector similarity search engine that achieves sub second query times. We will demonstrate architectural diagrams, showcasing how we handle millions of products with strict data isolation and achieve superior accuracy compared to standard implementations. We can show you how it works in one of customers - https://ro.vivre.eu/

- Event context: AI Tinkerers Meetup August 13 — 2025-08-13 — Berlin
- Public talk page: https://berlin.aitinkerers.org/talks/rsvp_7LBNQm7JTWM

### [Combined laser projection and AI Inspection for carbon fibre composites](https://waterloo.aitinkerers.org/talks/rsvp_KZQIFklfhmA)

The IRIS AI-powered Camera and Composite Inspection Software platform brings cutting-edge AI inspection to aerospace and automotive manufacturing, ensuring unmatched precision and productivity. Combining advanced laser projection with real-time insights, IRIS AI detects issues instantly, reduces rework, and seamlessly integrates with existing systems. Capture each step of production with high-resolution imaging and keep your quality control a step ahead. We will be presenting how our system implements a microservice architecture for quality assurance insde a customers air-gapped LAN. We'll talk about software architecture and the challenges of using AI alongside human operators.

- Event context: AI Tinkerers - Waterloo June Meetup — 2025-06-02 — Waterloo
- Public talk page: https://waterloo.aitinkerers.org/talks/rsvp_KZQIFklfhmA

### [Burn money or make GPU go brrr (and burn less money)](https://amsterdam.aitinkerers.org/talks/rsvp_igh8SwjkhzE)

When does it make sense to self-host an AI model yourself? Lets do some tokenomics and figure out when it makes sense to self-host. Then lets actual self-host a model with a SOTA inference setup powered by Aibrix or vllm-production stack. Can we achieve the performance we need to beat OpenAI on a Nebius mk8s? How does our choice of inference framework help us?

- Event context: AI Tinkerers Amsterdam - May Edition — 2025-05-06 — Amsterdam
- Public talk page: https://amsterdam.aitinkerers.org/talks/rsvp_igh8SwjkhzE

### [Distillation and fine-tuning of SLMs to achieve great performance in specific tasks](https://santiago.aitinkerers.org/talks/rsvp_KbjIwuA3MpA)

This process compresses large-scale language models into efficient, task-optimized versions, enhancing performance and adaptability. I will present a code, an example of performance and an example of the format of the training data

- Event context: AI Tinkerers - Santiago — 2025-02-27 — Santiago
- Public talk page: https://santiago.aitinkerers.org/talks/rsvp_KbjIwuA3MpA

### [Seguridad Infantil: El Uso de Cámaras de Video e IA para el Cuidado de Niños](https://bogota.aitinkerers.org/talks/rsvp_GHOgUtTJPL0)

El proyecto "Seguridad Infantil" tiene como meta principal el desarrollo de un sistema de monitoreo inteligente que emplea cámaras de video equipadas con inteligencia artificial (IA) para salvaguardar la seguridad y el bienestar de los niños. Este sistema está diseñado para ofrecer a padres y cuidadores una herramienta efectiva que les permita supervisar a los menores, detectar situaciones de riesgo y responder de manera ágil ante posibles emergencias. El proyecto incluye varios procesos clave, tales como: Redes Neuronales Convolucionales (CNN): Estas son fundamentales para la detección y clasificación de imágenes. Se emplearán técnicas de transferencia de aprendizaje, utilizando conjuntos de datos específicos para entrenar modelos que mejoren la precisión al identificar situaciones de riesgo. Análisis de Video en Tiempo Real: La implementación de IA permitirá analizar transmisiones de video en vivo, facilitando la identificación de comportamientos peligrosos en tiempo real. Análisis Predictivo: Se aplicarán modelos de aprendizaje automático que examinarán datos históricos para anticipar comportamientos de riesgo e identificar patrones relacionados con la victimización. Estos modelos se alimentarán con información sobre interacciones reales de peligro. Modelos de Redes Neuronales Recurrentes (RNN): Se utilizarán RNN para el análisis de series temporales, lo que permitirá comprender las variaciones en el comportamiento de un niño a lo largo del tiempo y detectar señales tempranas de riesgo. Privacidad Diferencial: Se garantizará que los sistemas de IA cumplan con normativas de protección de datos, como el GDPR o el COPPA, mediante la implementación de técnicas de privacidad diferencial. Esto permitirá a las organizaciones obtener información valiosa sin poner en riesgo la privacidad de los menores.

- Event context: AI Tinkerers Bogotá - Women Edition — 2024-11-28 — Bogotá
- Public talk page: https://bogota.aitinkerers.org/talks/rsvp_GHOgUtTJPL0

### [Efficient Gesture Recognition](https://boise.aitinkerers.org/talks/rsvp_HRIz6I8dXRY)

Efficient model for gesture recognition using Machine Vision

- Event context: AI Tinkerers - Boise October Meetup Announcement — 2024-10-29 — Boise
- Public talk page: https://boise.aitinkerers.org/talks/rsvp_HRIz6I8dXRY

### [Custom GPU-powered LLMs and Graph Neural Networks](https://boston.aitinkerers.org/talks/rsvp_LZl89lqLF3c)

A demo on running a variety of open LLMs with the power of GPUs. May dabble in using GPUs for graph neural networks trained on knowledge graphs.

- Event context: July 2024 Meetup at C10 Labs — 2024-07-22 — Boston
- Public talk page: https://boston.aitinkerers.org/talks/rsvp_LZl89lqLF3c

## Related Technologies

- [Python](https://aitinkerers.org/technologies/python) ([Markdown](https://aitinkerers.org/technologies/python.md)) — 662 public demos
- [Docker](https://aitinkerers.org/technologies/docker) ([Markdown](https://aitinkerers.org/technologies/docker.md)) — 147 public demos
- [PyTorch](https://aitinkerers.org/technologies/pytorch) ([Markdown](https://aitinkerers.org/technologies/pytorch.md)) — 273 public demos
- [GPT-4](https://aitinkerers.org/technologies/gpt-4) ([Markdown](https://aitinkerers.org/technologies/gpt-4.md)) — 529 public demos
- [OpenAI API](https://aitinkerers.org/technologies/openai-api) ([Markdown](https://aitinkerers.org/technologies/openai-api.md)) — 520 public demos
- [OpenCV](https://aitinkerers.org/technologies/opencv) ([Markdown](https://aitinkerers.org/technologies/opencv.md)) — 26 public demos
- [vLLM](https://aitinkerers.org/technologies/vllm) ([Markdown](https://aitinkerers.org/technologies/vllm.md)) — 33 public demos
- [BERT](https://aitinkerers.org/technologies/bert) ([Markdown](https://aitinkerers.org/technologies/bert.md)) — 179 public demos
- [BLOOM](https://aitinkerers.org/technologies/bloom) ([Markdown](https://aitinkerers.org/technologies/bloom.md)) — 115 public demos
- [Boost](https://aitinkerers.org/technologies/boost) ([Markdown](https://aitinkerers.org/technologies/boost.md)) — 1 public demo
- [Brode fireball evolution](https://aitinkerers.org/technologies/brode-fireball-evolution) ([Markdown](https://aitinkerers.org/technologies/brode-fireball-evolution.md)) — 1 public demo
- [C++](https://aitinkerers.org/technologies/c) ([Markdown](https://aitinkerers.org/technologies/c.md)) — 10 public demos
- [cuBLASLt](https://aitinkerers.org/technologies/cublaslt) ([Markdown](https://aitinkerers.org/technologies/cublaslt.md)) — 1 public demo
- [cudaMalloc](https://aitinkerers.org/technologies/cudamalloc) ([Markdown](https://aitinkerers.org/technologies/cudamalloc.md)) — 1 public demo
- [cudaMallocAsync](https://aitinkerers.org/technologies/cudamallocasync) ([Markdown](https://aitinkerers.org/technologies/cudamallocasync.md)) — 1 public demo
- [DGX Spark](https://aitinkerers.org/technologies/dgx-spark) ([Markdown](https://aitinkerers.org/technologies/dgx-spark.md)) — 1 public demo
- [diffprivlib](https://aitinkerers.org/technologies/diffprivlib) ([Markdown](https://aitinkerers.org/technologies/diffprivlib.md)) — 1 public demo
- [DirectX](https://aitinkerers.org/technologies/directx) ([Markdown](https://aitinkerers.org/technologies/directx.md)) — 1 public demo