# Video Understanding Projects at AI Tinkerers

> Canonical HTML: https://aitinkerers.org/technologies/video-understanding
> Markdown URL: https://aitinkerers.org/technologies/video-understanding.md
> Technology record last updated: 2026-02-26T02:31:06Z
> Generated: 2026-09-22T13:47:19Z

Video Understanding is the computer vision discipline that uses multimodal AI (e.g., VideoPrism) to interpret the temporal and spatial dynamics of video: extracting objects, actions, and context across frames.

This technology is a critical component of modern computer vision, moving beyond static image recognition to grasp the 'story' unfolding in a video clip. It leverages advanced deep learning models, like Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs), to analyze both spatial features (objects) and temporal features (motion, speed). Key applications include autonomous driving, where systems predict pedestrian behavior, and security surveillance, where it detects suspicious activity in real-time. For instance, foundation models like Google Research's VideoPrism achieve state-of-the-art results on 30 out of 33 benchmarks by training on massive datasets, including 36 million high-quality video-text pairs: that’s the scale required to truly understand video at a global level.

- Official technology site: https://twelvelabs.io/
- Public AI Tinkerers demos and talks: 3
- Result page: 1 of 1

## Recent Public Talks and Demos

### [How We Build Pipelines to Detect Product Issues and Fix Them with Coding Agents](https://prague.aitinkerers.org/talks/rsvp_C35CGsy9rLA)

I'll walk through how we built a pipeline that detects product issues from multiple data sources (session recordings, error logs, support tickets, LLM traces) and routes them to coding agents that generate PRs. * Step 1: Signal. Render videos from session recordings, transcribe them, and analyze, searching for issues. * Step 2: Cluster. Group signals from different sources and decide what’s worth fixing. * Step 3: Code. Preview of Twig ADE (AI Development Environment), where confirmed tasks get handed off to coding agents.

- Event context: Product Engineering: Apify x PostHog — 2026-02-26 — Prague
- Public talk page: https://prague.aitinkerers.org/talks/rsvp_C35CGsy9rLA

### [Video Understanding: Human Level Performance Using Multimodal Foundation Models with a Video First Ethos](https://denver-boulder.aitinkerers.org/talks/rsvp_6aXMaNtN0UE)

Video has traditionally be processed as its constituents: frames and audio but this isn't how humans process video. Learn how video foundation models take a video first approach to enabling human level understand of video.

- Event context: AI Tinkerers Denver - November Meetup — 2023-11-23 — Denver
- Public talk page: https://denver-boulder.aitinkerers.org/talks/rsvp_6aXMaNtN0UE

### [ML Generated Docs through Video Understanding](https://seattle.aitinkerers.org/talks/rsvp_Rd7nRlt5Gbs)

At Augmend we're developing ways to capture knowledge when devs work. We recently began experimenting with a new feature where we are able to auto-generate documentation just from "watching" your workflow. This allows for things like creating markdown for a how-to wiki without actually having to write anything or capturing the steps you took to solve a problem without taking notes yourself.

- Event context: AI Tinkerers Seattle - August Meetup — 2023-08-09 — Seattle
- Public talk page: https://seattle.aitinkerers.org/talks/rsvp_Rd7nRlt5Gbs

## Related Technologies

- [BERT](https://aitinkerers.org/technologies/bert) ([Markdown](https://aitinkerers.org/technologies/bert.md)) — 179 public demos
- [BLOOM](https://aitinkerers.org/technologies/bloom) ([Markdown](https://aitinkerers.org/technologies/bloom.md)) — 115 public demos
- [GPT-3](https://aitinkerers.org/technologies/gpt-3) ([Markdown](https://aitinkerers.org/technologies/gpt-3.md)) — 191 public demos
- [GPT-4](https://aitinkerers.org/technologies/gpt-4) ([Markdown](https://aitinkerers.org/technologies/gpt-4.md)) — 529 public demos
- [Llama-2](https://aitinkerers.org/technologies/llama-2) ([Markdown](https://aitinkerers.org/technologies/llama-2.md)) — 227 public demos
- [PaLM 2](https://aitinkerers.org/technologies/palm-2) ([Markdown](https://aitinkerers.org/technologies/palm-2.md)) — 116 public demos
- [RoBERTa](https://aitinkerers.org/technologies/roberta) ([Markdown](https://aitinkerers.org/technologies/roberta.md)) — 118 public demos
- [ABBYY FineReader](https://aitinkerers.org/technologies/abbyy-finereader) ([Markdown](https://aitinkerers.org/technologies/abbyy-finereader.md)) — 3 public demos
- [Agentic](https://aitinkerers.org/technologies/agentic) ([Markdown](https://aitinkerers.org/technologies/agentic.md)) — 3 public demos
- [Amazon Textract](https://aitinkerers.org/technologies/amazon-textract) ([Markdown](https://aitinkerers.org/technologies/amazon-textract.md)) — 5 public demos
- [Claude Code](https://aitinkerers.org/technologies/claude-code) ([Markdown](https://aitinkerers.org/technologies/claude-code.md)) — 216 public demos
- [Cloud Vision API](https://aitinkerers.org/technologies/cloud-vision-api) ([Markdown](https://aitinkerers.org/technologies/cloud-vision-api.md)) — 3 public demos
- [clustering](https://aitinkerers.org/technologies/clustering) ([Markdown](https://aitinkerers.org/technologies/clustering.md)) — 3 public demos
- [EasyOCR](https://aitinkerers.org/technologies/easyocr) ([Markdown](https://aitinkerers.org/technologies/easyocr.md)) — 2 public demos
- [Embeddings](https://aitinkerers.org/technologies/embeddings) ([Markdown](https://aitinkerers.org/technologies/embeddings.md)) — 22 public demos
- [Foundation models](https://aitinkerers.org/technologies/foundation-models) ([Markdown](https://aitinkerers.org/technologies/foundation-models.md)) — 5 public demos
- [Gemini 3](https://aitinkerers.org/technologies/gemini-3) ([Markdown](https://aitinkerers.org/technologies/gemini-3.md)) — 8 public demos
- [KMeans](https://aitinkerers.org/technologies/kmeans) ([Markdown](https://aitinkerers.org/technologies/kmeans.md)) — 2 public demos