# BLIP Projects at AI Tinkerers

> Canonical HTML: https://aitinkerers.org/technologies/blip
> Markdown URL: https://aitinkerers.org/technologies/blip.md
> Technology record last updated: 2026-02-25T04:12:26Z
> Generated: 2026-09-21T23:48:15Z

Salesforce's unified vision-language framework that uses synthetic data bootstrapping to outperform models trained on noisy web data.

BLIP (Bootstrapping Language-Image Pre-training) addresses the noise inherent in large-scale web datasets via a specialized CapFilt mechanism. This process uses a Captioner to generate synthetic labels and a Filter to prune low-quality matches (ensuring high-fidelity training data). The model dominates benchmarks like COCO and VQA (achieving a +2.7% boost in average recall@1 on COCO) by unifying vision-language understanding and generation into a single encoder-decoder framework. It provides a streamlined solution for image captioning, visual search, and zero-shot reasoning.

- Official technology site: https://github.com/salesforce/BLIP
- Public AI Tinkerers demos and talks: 4
- Result page: 1 of 1

## Recent Public Talks and Demos

### [Shop Talk](https://st-louis.aitinkerers.org/talks/rsvp_HCVQpW0TgEw)

I built ShopTalk, a multimodal shopping assistant that lets users search for products using text, images, or a combination of both, delivering more human-like and relevant results. In the demo, users can upload an image, type a query, or do both, and the system uses BLIP to generate contextual captions and CLIP to embed both text and images into a shared vector space, retrieving and ranking the most relevant products through a Redis-backed similarity search with late fusion scoring.

- Event context: AI Tinkerers St. Louis: April 2026 Meetup — 2026-04-14 — St. Louis
- Public talk page: https://st-louis.aitinkerers.org/talks/rsvp_HCVQpW0TgEw

### [classifai.dev - simple, self-improving classification api](https://la.aitinkerers.org/talks/rsvp_1OwmludKh3w)

The simplest possible classification API that any developer can use. Simply provide data you'd like classified, and the classes/labels OR a description of the task. If you provide feedback, the same endpoint starts improving its responses over time.

- Event context: AI Tinkerers LA – October 2025: Ghosts in the Machine w/ Oxen.ai — 2025-10-21 — Los Angeles
- Public talk page: https://la.aitinkerers.org/talks/rsvp_1OwmludKh3w

### [Browsing the web with AI](https://toronto.aitinkerers.org/talks/rsvp_mtJyKffdKIM)

Web scraping is broken. Companies spend millions maintaining brittle scrapers while developers waste countless hours rebuilding the same solutions. The emergence of powerful vision-language models (VLMs) and LLMs creates an opportunity to revolutionize this space. I'll demonstrate a novel architecture that combines: - Microsoft VLM for visual understanding and DOM parsing - Claude for reasoning and task planning - Selenium for browser automation - Custom prompt engineering for reliable structured output We'll explore: - Why traditional scrapers fail - How VLMs understand web interfaces - Prompt engineering for reliable agents - Live demo: Flight price comparison - Challenges in hallucination prevention - Open source architecture decisions Key technical innovations: - Vision-guided DOM traversal - RAG memory during browsing - Structured data extraction This project started from personal frustration with repetitive research tasks. The goal: make web automation accessible to everyone while being reliable enough for production use. Live demo will showcase the agent finding flight prices and returning structured JSON - all without human intervention.

- Event context: AI Tinkerers Toronto - January 2025 Meetup at Google **sold out** — 2025-01-30 — Toronto
- Public talk page: https://toronto.aitinkerers.org/talks/rsvp_mtJyKffdKIM

### [Applying 4o Vision Finetuning to Chemistry Diagrams](https://singapore.aitinkerers.org/talks/rsvp_7w9joR3W_oI)

The task is to extract student's attempts for chemistry diagram questions. These diagrams are a graph with nodes and edges. Using VLLMs out of the box often results in the model correcting the chemistry equations or missing key notation. Here we explore Vision Finetuning, and see how far we can go with less than 10 hand-labelled examples. Kuang Wen and I will show the data we have, the augmentation techniques, and our current demo app comparing finetuned with non-finetuned.

- Event context: AI Tinkerers Singapore: 3rd Meetup - November 19th, 2024 — 2024-11-19 — Singapore
- Public talk page: https://singapore.aitinkerers.org/talks/rsvp_7w9joR3W_oI

## Related Technologies

- [CLIP](https://aitinkerers.org/technologies/clip) ([Markdown](https://aitinkerers.org/technologies/clip.md)) — 10 public demos
- [BLIP-2](https://aitinkerers.org/technologies/blip-2) ([Markdown](https://aitinkerers.org/technologies/blip-2.md)) — 3 public demos
- [Flamingo](https://aitinkerers.org/technologies/flamingo) ([Markdown](https://aitinkerers.org/technologies/flamingo.md)) — 3 public demos
- [LXMERT](https://aitinkerers.org/technologies/lxmert) ([Markdown](https://aitinkerers.org/technologies/lxmert.md)) — 4 public demos
- [UNITER](https://aitinkerers.org/technologies/uniter) ([Markdown](https://aitinkerers.org/technologies/uniter.md)) — 3 public demos
- [ViLBERT](https://aitinkerers.org/technologies/vilbert) ([Markdown](https://aitinkerers.org/technologies/vilbert.md)) — 4 public demos
- [VisualBERT](https://aitinkerers.org/technologies/visualbert) ([Markdown](https://aitinkerers.org/technologies/visualbert.md)) — 3 public demos
- [ABBYY FineReader](https://aitinkerers.org/technologies/abbyy-finereader) ([Markdown](https://aitinkerers.org/technologies/abbyy-finereader.md)) — 3 public demos
- [Amazon Textract](https://aitinkerers.org/technologies/amazon-textract) ([Markdown](https://aitinkerers.org/technologies/amazon-textract.md)) — 5 public demos
- [Azure Computer Vision](https://aitinkerers.org/technologies/azure-computer-vision) ([Markdown](https://aitinkerers.org/technologies/azure-computer-vision.md)) — 1 public demo
- [BERT](https://aitinkerers.org/technologies/bert) ([Markdown](https://aitinkerers.org/technologies/bert.md)) — 179 public demos
- [BLOOM](https://aitinkerers.org/technologies/bloom) ([Markdown](https://aitinkerers.org/technologies/bloom.md)) — 115 public demos
- [Claude](https://aitinkerers.org/technologies/claude) ([Markdown](https://aitinkerers.org/technologies/claude.md)) — 174 public demos
- [Data Augmentation](https://aitinkerers.org/technologies/data-augmentation) ([Markdown](https://aitinkerers.org/technologies/data-augmentation.md)) — 1 public demo
- [Demo App](https://aitinkerers.org/technologies/demo-app) ([Markdown](https://aitinkerers.org/technologies/demo-app.md)) — 1 public demo
- [DOM parsing](https://aitinkerers.org/technologies/dom-parsing) ([Markdown](https://aitinkerers.org/technologies/dom-parsing.md)) — 1 public demo
- [FAISS](https://aitinkerers.org/technologies/faiss) ([Markdown](https://aitinkerers.org/technologies/faiss.md)) — 17 public demos
- [Fireworks AI](https://aitinkerers.org/technologies/fireworks-ai) ([Markdown](https://aitinkerers.org/technologies/fireworks-ai.md)) — 1 public demo