Reseachers at Google tackled the challenge of on-device low-power song recognition. The resulting Pixel 2 feature is convenient if you need it. In most cases it just makes me smile a bit whenever I glance at the locked screen with a song name on it.
But how does it work?
Low-power music detection
The Pixel 2 has a dedicated digital signal processor (DSP) which runs continuously a small music detector. The detector’s NN model has only 8 000 parameters and occupies less than 10 KB of memory. It’s only purpose is to decide if a song is playing or not.
Training
- Dataset: a subset of AudioSet 1 and noise-added audio clips
- Process: a random subclip from the dataset ‒> binary classification music present / not present
On-device music classification
Once the DSP has recognized music is playing, the application processor is woken up.
More soon…
Full paper: https://arxiv.org/pdf/1711.10958.pdf
References
-
J. F. Gemmeke, D. P. W. Ellis, D. Freedman, A. Jansen, W. Lawrence, R. C. Moore, M. Plakal, and M. Ritter. Audio set: An ontology and human-labeled dataset for audio events. In Proc. IEEE ICASSP 2017, New Orleans, LA, 2017. ↩