Technology

Mulan

Mulan is a joint acoustic-semantic embedding model that links music recordings to natural language descriptions using a dual-encoder architecture.

Developed by Google Research, Mulan maps unaligned audio and text into a shared 128-dimensional embedding space. The model leverages two distinct towers (a ResNet-50 for audio and a BERT-base for text) trained on 44 million music clips and 370,000 hours of audio. By utilizing contrastive learning, Mulan enables zero-shot music tagging and cross-modal retrieval without requiring manual annotations. This technology powers advanced music understanding tasks, allowing systems to identify complex genres or moods (e.g., 'lo-fi hip hop for studying') directly from raw waveforms.

https://google-research.github.io/seanet/mulan/

What builders pair with Mulan

Projects using both technologies. Select a pairing to see a project.

Built with Mulan

Apache Spark 1 shared project BERT 1 shared project BLOOM 1 shared project ECC 1 shared project fastXML 1 shared project GitHub 1 shared project

11 more pairings

GPT-3 1 shared project GPT-4 1 shared project In-Context Learning 1 shared project Llama-2 1 shared project PaLM 2 1 shared project prompt space 1 shared project RAkEL 1 shared project RoBERTa 1 shared project scikit-learn 1 shared project TensorFlow 1 shared project XML-CNN 1 shared project

Pairing: Apache Spark

Photo from the event — Event photo

In-Context Learning for Extreme Multi-Label Classification

Chicago · July 23, 2024

view project | view Apache Spark

Recent Talks & Demos

Showing 1-1 of 1

Members-Only

Sign in to see who built these projects

Sign in View FAQ

Extreme Multi-Label ICL