# DeiT Projects at AI Tinkerers

> Canonical HTML: https://aitinkerers.org/technologies/deit
> Markdown URL: https://aitinkerers.org/technologies/deit.md
> Technology record last updated: 2026-04-24T15:29:41Z
> Generated: 2026-09-22T21:34:16Z

DeiT (Data-efficient Image Transformers) is a Vision Transformer (ViT) variant that achieves high-accuracy image classification using a novel distillation token, cutting training time and data requirements significantly.

DeiT, introduced by Hugo Touvron et al., solves the original Vision Transformer's (ViT) data hunger: it delivers competitive, convolution-free performance without massive external pre-training datasets. The core innovation is a **distillation token**—a teacher-student strategy that efficiently transfers inductive bias, often from a ConvNet teacher, to the ViT architecture. This method allows training on ImageNet-1k only, completing the process in less than three days on a single machine. The result: the distilled model hits up to **85.2% top-1 accuracy**, making high-performance visual transformers accessible and resource-efficient.

- Official technology site: https://huggingface.co/docs/transformers/model_doc/deit
- Public AI Tinkerers demos and talks: 1
- Result page: 1 of 1

## Recent Public Talks and Demos

### [Modelo de clasificación de imágenes basado en Transformers para la evaluación de daños estructurales post-sísmicos de acuerdo con la Escala Macrosísmica europea: comparativa con técnicas de aprendizaje de máquina](https://quito.aitinkerers.org/talks/rsvp_IxDDfiiTmsU)

Fue mi tema de tesis consistió en comparar arquitecturas tradicionales de redes neuronales convolucionales (CNN) con modelos basados en Transformers, específicamente DeiT (Data-efficient Image Transformers), para la clasificación automática de imágenes de estructuras afectadas por sismos. Se trabajó con un dataset de 1500 imágenes (750 de mampostería y 750 de hormigón armado), evaluando el daño estructural conforme a la Escala Macrosísmica Europea (EMS-98). Se aplicaron técnicas de data aumentaron. La comparación se hizo con una tesis previa que usaba modelos CNN como VGG16, DenseNet121, MobileNetV2, entre otros, para contrastarlos con el desempeño de DeiT.

- Event context: AI Tinkerers - Quito Primer Meetup (Abril) — 2025-04-24 — Quito
- Public talk page: https://quito.aitinkerers.org/talks/rsvp_IxDDfiiTmsU

## Related Technologies

- [DenseNet121](https://aitinkerers.org/technologies/densenet121) ([Markdown](https://aitinkerers.org/technologies/densenet121.md)) — 1 public demo
- [Google Colab](https://aitinkerers.org/technologies/google-colab) ([Markdown](https://aitinkerers.org/technologies/google-colab.md)) — 12 public demos
- [Python](https://aitinkerers.org/technologies/python) ([Markdown](https://aitinkerers.org/technologies/python.md)) — 662 public demos
- [PyTorch](https://aitinkerers.org/technologies/pytorch) ([Markdown](https://aitinkerers.org/technologies/pytorch.md)) — 273 public demos