Can AI truly edit audio, not just generate it? 🎧

Tencent Hy, in collaboration with SJTU, SII, NTU, TJU, ZODA, PKU, FDU, and other collaborators, introduces MMAE.

MMAE--A Massive Multitask Audio Editing Benchmark, is the first comprehensive evaluation benchmark for speech and audio "Banana🍌"

Instead of simply requiring the AI to "generate" audio, it demands that the AI understand an existing audio clip and precisely modify it according to natural language instructions—altering what needs to be changed while leaving the rest untouched.

Current models show an Exact Match Rate (EMR) below 5%, revealing a major gap in reliable audio editing.

MMAE includes:

✅ 2,000 high-fidelity samples from real-world scenarios

✅ 17,741 fine-grained rubric evaluation items

✅ 7 modality settings across sound, music, speech and their mixtures

✅ 6 task complexity from basic modifications to multi-hop reasoning and multi-round editing

✅ 8 operation types across local and global granularities

How to use:

arXiv: arxiv.org/abs/2606.07229

GitHub: github.com/ddlBoJack/MMAE

HuggingFace: huggingface.co/datasets/BoJa…

Demo: piped.video/6At5nTWhlXI

Video