# Set of Marks Projects at AI Tinkerers

> Canonical HTML: https://aitinkerers.org/technologies/set-of-marks
> Markdown URL: https://aitinkerers.org/technologies/set-of-marks.md
> Technology record last updated: 2026-03-03T01:10:32Z
> Generated: 2026-10-15T07:18:43Z

Set-of-Mark (SoM) is a visual prompting technique: it segments images and overlays alphanumeric marks to dramatically enhance Large Multimodal Models' (LMMs) visual grounding capabilities.

SoM is a novel visual prompting method designed to unleash the full visual grounding potential of LMMs, specifically GPT-4V. The process is direct: an off-the-shelf segmentation model (e.g., SEEM or SAM) partitions an image into distinct regions. We then overlay these regions with a set of speakable marks (alphanumerics, masks, or boxes). This marked image, when input to the LMM, provides explicit spatial and object relationship context, which was previously a blind spot. Empirical studies confirm SoM's effectiveness: it enables GPT-4V to outperform state-of-the-art, fully-finetuned models on fine-grained vision tasks like RefCOCOg, all in a zero-shot setting.

- Official technology site: https://github.com/microsoft/SoM
- Public AI Tinkerers demos and talks: 1
- Result page: 1 of 1

## Recent Public Talks and Demos

### [Multimodal Maestro with GPT-4 Vision](https://sf.aitinkerers.org/talks/rsvp_5dtCu0fp6a8)

- Event context: AI Tinkerers SF X Cerebral Valley - Dec Meetup — 2023-12-04 — San Francisco
- Public talk page: https://sf.aitinkerers.org/talks/rsvp_5dtCu0fp6a8

## Related Technologies

- [CogLVM](https://aitinkerers.org/technologies/coglvm) ([Markdown](https://aitinkerers.org/technologies/coglvm.md)) — 1 public demo
- [GPT-4 Vision](https://aitinkerers.org/technologies/gpt-4-vision) ([Markdown](https://aitinkerers.org/technologies/gpt-4-vision.md)) — 2 public demos
- [LLaVA](https://aitinkerers.org/technologies/llava) ([Markdown](https://aitinkerers.org/technologies/llava.md)) — 5 public demos
- [Segment Anything Model](https://aitinkerers.org/technologies/segment-anything-model) ([Markdown](https://aitinkerers.org/technologies/segment-anything-model.md)) — 5 public demos