# XLNet Projects at AI Tinkerers
> Canonical HTML: https://aitinkerers.org/technologies/xlnet
> Markdown URL: https://aitinkerers.org/technologies/xlnet.md
> Technology record last updated: 2026-03-02T16:01:25Z
> Generated: 2026-09-22T05:46:07Z
A generalized autoregressive pretraining method that outperforms BERT by leveraging permutation language modeling to capture bidirectional context without masking noise.
Developed by researchers at Google Brain and Carnegie Mellon University, XLNet integrates the best features of autoregressive (AR) and autoencoding (AE) models. By utilizing Permutation Language Modeling (PLM), it predicts tokens across all possible factorizations of a sequence order, effectively bypassing the artificial [MASK] tokens that hinder BERT during fine-tuning. This architecture incorporates Transformer-XL mechanisms (segment-level recurrence and relative positional encoding) to handle long-range dependencies. In benchmarks, XLNet outperformed BERT on 20 tasks, achieving state-of-the-art results in SQuAD 2.0 and GLUE by modeling bidirectional context while maintaining the mathematical consistency of traditional language models.
- Official technology site: https://arxiv.org/abs/1906.08237
- Public AI Tinkerers demos and talks: 3
- Result page: 1 of 1
## Recent Public Talks and Demos
### [Tackling the ARC challenge as a side quest](https://london.aitinkerers.org/talks/rsvp_bKwAmYkOurk)
The Abstraction and Reasoning Corpus, designed by François Chollet, is a set of visual problems, known to be one of the hardest reasoning benchmarks which state of the art LLMs fail at. We'll be sharing our work and learnings from tackling the ARC challenge (arcprize.org) during the summer, and how we applied the intuition behind our reasoning engine to this benchmark. We will be open-sourcing a couple of elements including the solver and dataset, which we will present during that demo.
- Event context: AI Tinkerers London - December Meetup π β 2024-12-04 β London
- Public talk page: https://london.aitinkerers.org/talks/rsvp_bKwAmYkOurk
### [Enhancing AI with RAG - Techniques to improve accuracy](https://bogota.aitinkerers.org/talks/rsvp_QjI40TNRTXw)
This talk explores how to optimize AI companions for project management using RAG. Standard RAG models struggle with context, leading to incomplete or inaccurate data retrieval, which is important when managing resources and making decisions. Weβll explore GraphRAG and context-aware RAG, which use knowledge graphs and enhanced contextual embeddings to improve the precision and relevance of information retrieval.
- Event context: AI Tinkerers Octubre en Colombia 4.0 β 2024-10-30 β BogotΓ‘
- Public talk page: https://bogota.aitinkerers.org/talks/rsvp_QjI40TNRTXw
### [Deepgram (Community Demo Pit)](https://seattle.aitinkerers.org/talks/rsvp_MZOrhBz-sLA)
Stop by the community demo pit to check out amazing demos from Deepgram! Deepgram is a pioneering voice AI platform offering cutting-edge APIs for speech-to-text, text-to-speech, and language understanding. Their lightning-fast, highly accurate speech recognition and natural-sounding voice synthesis enable developers to build sophisticated voice AI applications with ease. Deepgramβs GPU-optimized infrastructure provides unparalleled speed and cost-efficiency, making it up to 40x faster and 3-5x cheaper than alternatives. From conversational AI to medical transcription, Deepgram empowers developers to unlock the full potential of voice data at scale.
- Event context: AI Tinkerers Summer Social - August 2024 β 2024-08-16 β Seattle
- Public talk page: https://seattle.aitinkerers.org/talks/rsvp_MZOrhBz-sLA
## Related Technologies
- [BERT](https://aitinkerers.org/technologies/bert) ([Markdown](https://aitinkerers.org/technologies/bert.md)) β 179 public demos
- [RoBERTa](https://aitinkerers.org/technologies/roberta) ([Markdown](https://aitinkerers.org/technologies/roberta.md)) β 118 public demos
- [alBERT](https://aitinkerers.org/technologies/albert) ([Markdown](https://aitinkerers.org/technologies/albert.md)) β 4 public demos
- [GPT-3](https://aitinkerers.org/technologies/gpt-3) ([Markdown](https://aitinkerers.org/technologies/gpt-3.md)) β 191 public demos
- [GPT-4](https://aitinkerers.org/technologies/gpt-4) ([Markdown](https://aitinkerers.org/technologies/gpt-4.md)) β 529 public demos
- [T5](https://aitinkerers.org/technologies/t5) ([Markdown](https://aitinkerers.org/technologies/t5.md)) β 5 public demos
- [arcprize](https://aitinkerers.org/technologies/arcprize) ([Markdown](https://aitinkerers.org/technologies/arcprize.md)) β 1 public demo
- [BART](https://aitinkerers.org/technologies/bart) ([Markdown](https://aitinkerers.org/technologies/bart.md)) β 4 public demos
- [Conversational AI](https://aitinkerers.org/technologies/conversational-ai) ([Markdown](https://aitinkerers.org/technologies/conversational-ai.md)) β 5 public demos
- [Deepgram](https://aitinkerers.org/technologies/deepgram) ([Markdown](https://aitinkerers.org/technologies/deepgram.md)) β 11 public demos
- [ELMo](https://aitinkerers.org/technologies/elmo) ([Markdown](https://aitinkerers.org/technologies/elmo.md)) β 1 public demo
- [GPT-2](https://aitinkerers.org/technologies/gpt-2) ([Markdown](https://aitinkerers.org/technologies/gpt-2.md)) β 4 public demos
- [Graph database](https://aitinkerers.org/technologies/graph-database) ([Markdown](https://aitinkerers.org/technologies/graph-database.md)) β 2 public demos
- [GraphRAG](https://aitinkerers.org/technologies/graphrag) ([Markdown](https://aitinkerers.org/technologies/graphrag.md)) β 13 public demos
- [Knowledge Graph](https://aitinkerers.org/technologies/knowledge-graph) ([Markdown](https://aitinkerers.org/technologies/knowledge-graph.md)) β 9 public demos
- [LLMs](https://aitinkerers.org/technologies/llms) ([Markdown](https://aitinkerers.org/technologies/llms.md)) β 83 public demos
- [LUIS](https://aitinkerers.org/technologies/luis) ([Markdown](https://aitinkerers.org/technologies/luis.md)) β 1 public demo
- [NLTK](https://aitinkerers.org/technologies/nltk) ([Markdown](https://aitinkerers.org/technologies/nltk.md)) β 4 public demos