# TF-IDF Projects at AI Tinkerers

> Canonical HTML: https://aitinkerers.org/technologies/tf-idf
> Markdown URL: https://aitinkerers.org/technologies/tf-idf.md
> Technology record last updated: 2026-03-04T09:23:37Z
> Generated: 2026-09-22T03:43:43Z

TF-IDF (Term Frequency-Inverse Document Frequency) is a statistical measure that quantifies a term's relevance in a document by multiplying its local frequency (TF) with its global rarity (IDF).

TF-IDF is a core statistical method in information retrieval and text mining: it assigns a numerical weight to a word, signaling its importance within a document relative to a larger corpus. The calculation is direct: Term Frequency (TF) measures how often a word appears in the document, and Inverse Document Frequency (IDF) scales that value down if the word (like 'the' or 'a') is common across all documents. The final TF-IDF score emphasizes terms that are frequent in a specific document but rare overall (e.g., 'quantum' in a physics paper). This vectorization process is crucial for applications like building search engine relevance rankings and training machine learning models for text classification.

- Official technology site: https://www.capitalone.com/tech/machine-learning/understanding-tf-idf-for-machine-learning/
- Public AI Tinkerers demos and talks: 3
- Result page: 1 of 1

## Recent Public Talks and Demos

### [Más allá del RAG: Grafos universales para datos no universales](https://santiago.aitinkerers.org/talks/rsvp_Ic5kPKxkc3I)

Hoy, la mayoría de los sistemas de information retrieval usan arquitecturas tipo RAG o GraphRAG: se basan en búsqueda semántica, embeddings y modelos de lenguaje para recuperar información relevante.

- Event context: Estructura, Risa y Ecosistemas: el nuevo ADN de la IA Chilena — 2025-10-29 — Santiago
- Public talk page: https://santiago.aitinkerers.org/talks/rsvp_Ic5kPKxkc3I

### [Building Magic the gathering decks using AI](https://medellin.aitinkerers.org/talks/rsvp_OmejSCJ_mo4)

Back to basics, no LLM based solutions. A small project where I gather data from public sources to build custom decks. Understanding the logic behind basic tf/idf and usual magic deck building we can create customs decks and iterate easily. The idea is simple: Calculate a “synergy” score between all pairs of cards and then optimize to create the deck with the most synergy

- Event context: AI Tinkerers Medellín #9 - 29 de Enero 2025 — 2025-01-29 — Medellín
- Public talk page: https://medellin.aitinkerers.org/talks/rsvp_OmejSCJ_mo4

### [Covariate Search](https://hong-kong.aitinkerers.org/talks/rsvp_bHTSKW0CfRo)

The first practical application of covariate search (with the potential of revolutionizing the search industry) by introducing a new modality. In the demo I am showcasing how we can vectorize a set of keywords semantically and perform a search on other sets (this is not currently possible with semantic search).

- Event context: AI Tinkerers - Hong Kong Meetup (December) - Inauguration — 2024-12-19 — Hong Kong
- Public talk page: https://hong-kong.aitinkerers.org/talks/rsvp_bHTSKW0CfRo

## Related Technologies

- [Apache Lucene](https://aitinkerers.org/technologies/apache-lucene) ([Markdown](https://aitinkerers.org/technologies/apache-lucene.md)) — 2 public demos
- [BM25](https://aitinkerers.org/technologies/bm25) ([Markdown](https://aitinkerers.org/technologies/bm25.md)) — 5 public demos
- [Elasticsearch](https://aitinkerers.org/technologies/elasticsearch) ([Markdown](https://aitinkerers.org/technologies/elasticsearch.md)) — 5 public demos
- [Embeddings](https://aitinkerers.org/technologies/embeddings) ([Markdown](https://aitinkerers.org/technologies/embeddings.md)) — 22 public demos
- [FAISS](https://aitinkerers.org/technologies/faiss) ([Markdown](https://aitinkerers.org/technologies/faiss.md)) — 17 public demos
- [HNSW](https://aitinkerers.org/technologies/hnsw) ([Markdown](https://aitinkerers.org/technologies/hnsw.md)) — 7 public demos
- [Solr](https://aitinkerers.org/technologies/solr) ([Markdown](https://aitinkerers.org/technologies/solr.md)) — 2 public demos
- [Terrier](https://aitinkerers.org/technologies/terrier) ([Markdown](https://aitinkerers.org/technologies/terrier.md)) — 2 public demos
- [BERT](https://aitinkerers.org/technologies/bert) ([Markdown](https://aitinkerers.org/technologies/bert.md)) — 179 public demos
- [Covariate search](https://aitinkerers.org/technologies/covariate-search) ([Markdown](https://aitinkerers.org/technologies/covariate-search.md)) — 1 public demo
- [GPT-3](https://aitinkerers.org/technologies/gpt-3) ([Markdown](https://aitinkerers.org/technologies/gpt-3.md)) — 191 public demos
- [GPT-4](https://aitinkerers.org/technologies/gpt-4) ([Markdown](https://aitinkerers.org/technologies/gpt-4.md)) — 529 public demos
- [GraphRAG](https://aitinkerers.org/technologies/graphrag) ([Markdown](https://aitinkerers.org/technologies/graphrag.md)) — 13 public demos
- [Keras](https://aitinkerers.org/technologies/keras) ([Markdown](https://aitinkerers.org/technologies/keras.md)) — 74 public demos
- [Language Models](https://aitinkerers.org/technologies/language-models) ([Markdown](https://aitinkerers.org/technologies/language-models.md)) — 4 public demos
- [ONNX](https://aitinkerers.org/technologies/onnx) ([Markdown](https://aitinkerers.org/technologies/onnx.md)) — 83 public demos
- [PyTorch](https://aitinkerers.org/technologies/pytorch) ([Markdown](https://aitinkerers.org/technologies/pytorch.md)) — 273 public demos
- [RAG](https://aitinkerers.org/technologies/rag) ([Markdown](https://aitinkerers.org/technologies/rag.md)) — 147 public demos