# Semantic tokens Projects at AI Tinkerers

> Canonical HTML: https://aitinkerers.org/technologies/semantic-tokens
> Markdown URL: https://aitinkerers.org/technologies/semantic-tokens.md
> Technology record last updated: 2026-02-28T01:02:08Z
> Generated: 2026-10-25T15:41:32Z

Semantic tokens provide compiler-level symbol classification to code editors, enabling intelligent, context-aware syntax highlighting that transcends basic regex matching.

Semantic tokens deliver a critical layer of accuracy to code editors, moving beyond the limitations of standard TextMate grammars which rely on lexical, regex-based rules. This technology leverages the Language Server Protocol (LSP) to transmit rich, project-contextual data from the language service to the editor. For example, a standard grammar might color all `variable` names the same; a semantic token, however, can correctly distinguish a local `parameter` from a global `class` or `function` declaration. This compiler-informed understanding allows themes to apply precise styling, significantly improving code readability and developer experience.

- Official technology site: https://code.visualstudio.com/api/language-extensions/semantic-highlight-guide
- Public AI Tinkerers demos and talks: 1
- Result page: 1 of 1

## Recent Public Talks and Demos

### [Llama3s Speech Understanding - v2.0 improvements](https://singapore.aitinkerers.org/talks/rsvp_cUXd3XJ4yX8)

- Event context: AI Tinkerers Singapore: Meetup - September 16th, 2024 — 2024-09-16 — Singapore
- Public talk page: https://singapore.aitinkerers.org/talks/rsvp_cUXd3XJ4yX8

## Related Technologies

- [Early fusion](https://aitinkerers.org/technologies/early-fusion) ([Markdown](https://aitinkerers.org/technologies/early-fusion.md)) — 1 public demo
- [Hugging Face](https://aitinkerers.org/technologies/hugging-face) ([Markdown](https://aitinkerers.org/technologies/hugging-face.md)) — 42 public demos
- [Llama3-S](https://aitinkerers.org/technologies/llama3-s) ([Markdown](https://aitinkerers.org/technologies/llama3-s.md)) — 1 public demo
- [Multimodal AI](https://aitinkerers.org/technologies/multimodal-ai) ([Markdown](https://aitinkerers.org/technologies/multimodal-ai.md)) — 10 public demos