# HNSW Projects at AI Tinkerers

> Canonical HTML: https://aitinkerers.org/technologies/hnsw
> Markdown URL: https://aitinkerers.org/technologies/hnsw.md
> Technology record last updated: 2026-02-23T08:29:32Z
> Generated: 2026-09-22T01:44:43Z

HNSW (Hierarchical Navigable Small World) is a state-of-the-art graph-based algorithm: it executes Approximate Nearest Neighbor (ANN) search on high-dimensional vectors with logarithmic complexity (O(log n)), ensuring lightning-fast similarity retrieval.

Hierarchical Navigable Small World (HNSW) is the dominant Approximate Nearest Neighbor (ANN) search algorithm, delivering superior speed and recall for vector databases. It constructs a multi-layer proximity graph: higher layers contain long-range connections for rapid traversal, while lower layers provide fine-grained accuracy for finding the true nearest neighbors. This hierarchical structure, detailed in the 2016 paper by Malkov and Yashunin, achieves logarithmic complexity scaling, making it highly efficient. Use it to power critical applications like large-scale image retrieval, real-time product recommendation engines, and modern Retrieval-Augmented Generation (RAG) systems.

- Official technology site: https://arxiv.org/abs/1603.09320
- Public AI Tinkerers demos and talks: 7
- Result page: 1 of 1

## Recent Public Talks and Demos

### [Semantic Data Deduplication: HNSW, MNN, and LLM](https://montreal.aitinkerers.org/talks/rsvp_3hHtv5BiXF4)

I built a semantic deduplication pipeline designed to identify, validate, and merge duplicate event records across noisy datasets. The system operates in three layers: Vector Indexing (HNSW), Mutual Nearest Neighbors, and Agentic Reconciliation (LLM) ensuring no data is lost by aggregating all unique attributes and data points across sources.

- Event context: AI Tinkerers Montreal - March Demo Night — 2026-03-24 — Montreal
- Public talk page: https://montreal.aitinkerers.org/talks/rsvp_3hHtv5BiXF4

### [EmergentDB: How Evolutionary Algorithms Made My Vector Database 82x Faster](https://singapore.aitinkerers.org/talks/rsvp_hHLd9-vXn8A)

Vector databases power semantic search, RAG, and recommendation systems—but tuning them is a nightmare. Should HNSW use M=8 or M=32? What's the right ef_construction value? Most teams guess and pray. EmergentDB solves this through evolutionary computation. Instead of manual tuning, it uses the MAP-Elites algorithm to evolve the optimal index configuration for your specific workload. The system maintains a "quality-diversity grid" that explores the trade-off space between recall, latency, and memory—automatically selecting between HNSW, Flat, and IVF indices with evolved hyperparameters.

- Event context: AI Tinkerers - The Age of AI & Infrastructure (Singapore) — 2026-02-11 — Singapore
- Public talk page: https://singapore.aitinkerers.org/talks/rsvp_hHLd9-vXn8A

### [Más allá del RAG: Grafos universales para datos no universales](https://santiago.aitinkerers.org/talks/rsvp_Ic5kPKxkc3I)

Hoy, la mayoría de los sistemas de information retrieval usan arquitecturas tipo RAG o GraphRAG: se basan en búsqueda semántica, embeddings y modelos de lenguaje para recuperar información relevante.

- Event context: Estructura, Risa y Ecosistemas: el nuevo ADN de la IA Chilena — 2025-10-29 — Santiago
- Public talk page: https://santiago.aitinkerers.org/talks/rsvp_Ic5kPKxkc3I

### [Covariate Search](https://hong-kong.aitinkerers.org/talks/rsvp_bHTSKW0CfRo)

The first practical application of covariate search (with the potential of revolutionizing the search industry) by introducing a new modality. In the demo I am showcasing how we can vectorize a set of keywords semantically and perform a search on other sets (this is not currently possible with semantic search).

- Event context: AI Tinkerers - Hong Kong Meetup (December) - Inauguration — 2024-12-19 — Hong Kong
- Public talk page: https://hong-kong.aitinkerers.org/talks/rsvp_bHTSKW0CfRo

### [Nixiesearch: a new Lucene-based hybrid search engine which fine-tunes to your data](https://berlin.aitinkerers.org/talks/rsvp_jtED_DXuUG8)

Building a new search engine in 2024 sounds like a stupid idea: a new vector search startup is created every week, so how can you be different from so many competitors? But in practice, you can quickly discover that putting vectors into an HNSW index is not the most challenging part of building a search application that your customers would use and like. Relevance tuning, multi-field search, facets, filters, autocomplete suggestions - the RAG-vector search crowd is still discovering all these “novel” things. In this talk, we’re going to introduce Nixiesearch, an open-source hybrid search engine focused on solving typical search problems: * Based on Lucene. You get filters, facets, autocomplete, and complex queries out of the box with decent performance. * Cloud-native and serverless. Can use S3-compatible block storage for index persistence, being able to scale to zero. * Can fine-tune the underlying embedding model on your relevance labels (if you have them) or LLM-generated synthetic labels. Nixiesearch is still in an early development stage, so your opinion on “how to do a search engine right” is really important.

- Event context: AI Tinkerers Berlin - March 21 — 2024-03-21 — Berlin
- Public talk page: https://berlin.aitinkerers.org/talks/rsvp_jtED_DXuUG8

### [Scale your AI apps to millions of rows with pg_embedding](https://seattle.aitinkerers.org/talks/rsvp__Dhv_fahqR4)

Scaling databases is hard. This remains true even for vector databases. Nearly 50% of professional developers use Postgres, and many of them are building LLMs apps. The issue is that it is hard to scale with pgvector because of limitations around IVF index. As a response, we built and open sourced pg_embedding, which implements HNSW index in Postgres and performs 20x better than pgvector, to help developers scale their AI apps to millions of rows. In this talk, we will see how pg_embedding and its serverless driver perform against some well-known vector databases.

- Event context: AI Tinkerers Seattle - July Meetup — 2023-07-27 — Seattle
- Public talk page: https://seattle.aitinkerers.org/talks/rsvp__Dhv_fahqR4

### [Postgres similarity search with hnsw](https://sf.aitinkerers.org/talks/rsvp_tbtyvWZTZX4)

Postgres extension for approximate similarity search with hnsw index

- Event context: AI Tinkerers SF July 6th - RSVP REQUIRED — 2023-07-07 — San Francisco
- Public talk page: https://sf.aitinkerers.org/talks/rsvp_tbtyvWZTZX4

## Related Technologies

- [Apache Lucene](https://aitinkerers.org/technologies/apache-lucene) ([Markdown](https://aitinkerers.org/technologies/apache-lucene.md)) — 2 public demos
- [BM25](https://aitinkerers.org/technologies/bm25) ([Markdown](https://aitinkerers.org/technologies/bm25.md)) — 5 public demos
- [Elasticsearch](https://aitinkerers.org/technologies/elasticsearch) ([Markdown](https://aitinkerers.org/technologies/elasticsearch.md)) — 5 public demos
- [Embeddings](https://aitinkerers.org/technologies/embeddings) ([Markdown](https://aitinkerers.org/technologies/embeddings.md)) — 22 public demos
- [FAISS](https://aitinkerers.org/technologies/faiss) ([Markdown](https://aitinkerers.org/technologies/faiss.md)) — 17 public demos
- [pgvector](https://aitinkerers.org/technologies/pgvector) ([Markdown](https://aitinkerers.org/technologies/pgvector.md)) — 26 public demos
- [PostgreSQL](https://aitinkerers.org/technologies/postgresql) ([Markdown](https://aitinkerers.org/technologies/postgresql.md)) — 106 public demos
- [RAG](https://aitinkerers.org/technologies/rag) ([Markdown](https://aitinkerers.org/technologies/rag.md)) — 147 public demos
- [Solr](https://aitinkerers.org/technologies/solr) ([Markdown](https://aitinkerers.org/technologies/solr.md)) — 2 public demos
- [Terrier](https://aitinkerers.org/technologies/terrier) ([Markdown](https://aitinkerers.org/technologies/terrier.md)) — 2 public demos
- [TF-IDF](https://aitinkerers.org/technologies/tf-idf) ([Markdown](https://aitinkerers.org/technologies/tf-idf.md)) — 3 public demos
- [Amazon S3](https://aitinkerers.org/technologies/amazon-s3) ([Markdown](https://aitinkerers.org/technologies/amazon-s3.md)) — 8 public demos
- [ARM NEON](https://aitinkerers.org/technologies/arm-neon) ([Markdown](https://aitinkerers.org/technologies/arm-neon.md)) — 1 public demo
- [Cloudian HyperStore](https://aitinkerers.org/technologies/cloudian-hyperstore) ([Markdown](https://aitinkerers.org/technologies/cloudian-hyperstore.md)) — 1 public demo
- [Covariate search](https://aitinkerers.org/technologies/covariate-search) ([Markdown](https://aitinkerers.org/technologies/covariate-search.md)) — 1 public demo
- [DigitalOcean Spaces](https://aitinkerers.org/technologies/digitalocean-spaces) ([Markdown](https://aitinkerers.org/technologies/digitalocean-spaces.md)) — 1 public demo
- [EmergentDB](https://aitinkerers.org/technologies/map-elites-evolutionary-algorithm-index-types-hnsw) ([Markdown](https://aitinkerers.org/technologies/map-elites-evolutionary-algorithm-index-types-hnsw.md)) — 1 public demo
- [Flat (brute-force)](https://aitinkerers.org/technologies/flat-brute-force) ([Markdown](https://aitinkerers.org/technologies/flat-brute-force.md)) — 1 public demo