A tune is stuck in your head, but not its name, its words, or who sings it. Google’s answer is to let you hum it. Hum to Search, launched in October 2020, identifies a song from a 10 to 15 second hum, whistle or sung melody, with no lyrics and no perfect pitch required. This guide explains how to use it, why humming breaks ordinary song identification, and the machine learning that makes the match.
The short version is that Google does not listen for the song. It listens for the melody, converts it into a number-based sequence, and compares that sequence with the melodies of studio recordings. Instruments, vocal tone and background noise are stripped away, because none of them are the melody. What remains is a fingerprint of the tune itself, and that fingerprint is what gets matched.
There is a reason the feature caught on so fast. A melody stuck in your head, an earworm, is famously hard to shake, and research Google cites has found that engaging with the original song, by listening to it or singing it, helps drive the earworm away. Hum to Search is the shortcut to that original song when all you have is the tune.
How to Hum a Song to Google
Using it takes seconds. Open the latest version of the Google app on your phone, or find the Google Search widget, tap the mic icon and say “what’s this song?” or tap the “Search a song” button. Then hum, whistle or sing the melody for 10 to 15 seconds. On Google Assistant, say “Hey Google, what’s this song?” and then hum the tune.
Google shows the most likely matches based on the tune, with match percentages to help you judge. Pick the best match and you can explore information on the song and artist, watch accompanying music videos, listen on your favourite music app, find the lyrics, read analysis, and check out other recordings of the song when available.
At launch, the feature was available in English on iOS and in more than 20 languages on Android, with Google saying it hoped to expand to more languages. The experience today lives in the same places: the Google app’s microphone and the Assistant, both built around that same 10 to 15 second hum.
Why Humming Breaks Normal Song Identification
Identifying a song that is actually playing is a solved problem. The phone hears the recording itself, with its instruments, vocals and production intact, and matches it against a catalogue of recordings. Google’s own earlier systems did exactly this: Now Playing, released on Pixel phones in 2017, recognised songs on-device without a server connection, and Sound Search, added to the Google app in 2018, offered server-based recognition across more than 100 million songs.
A hum breaks that approach completely. A hummed tune contains far less information than a studio recording: no instruments, no backing vocals, no lyrics, no production, and no singer’s voice as recorded. Google illustrates the gap with spectrograms of a hummed “Bella Ciao” beside the studio recording: the hum is a single thin melody line, while the recording is a dense wall of sound.
Human humming adds a second problem: variation. By mistake or design, when someone hums their interpretation of a song, the pitch, key, tempo or rhythm may vary slightly or even significantly. The system cannot demand an exact match, because the input is never exact. It has to recognise the melody through all the ways people get it wrong.
That is why older approaches to this problem matched hums against databases of melody-only or hummed versions of songs, rather than against studio recordings directly. Those databases were limited and needed manual updating, which is why they never scaled. Google’s approach was to match the hum directly to the original polyphonic recording, with no hummed or MIDI version of each track required.
Two Different Identification Problems
It helps to see recorded identification and hum identification as two different problems, not one problem with a harder input. When a song is playing, the system hears the finished product: the singer’s voice, the instruments, the mix, all exactly as recorded. Matching means finding the recording that sounds like the recording, and the full audio gives the matcher a great deal to hold on to.
A hum offers almost none of that. It is one voice, producing one pitch line, with no accompaniment and no words. The system cannot compare sounds, because the sounds are nothing alike. It has to compare melodies, which is a more abstract thing: the pattern of notes, their order and their relative spacing, regardless of the voice that carries them.
That abstraction is also what makes the feature useful beyond perfect singers. A child humming a cartoon theme, a commuter whistling a chorus, and a singer performing the verse all produce wildly different audio, but they share the same melody. A system built on melody rather than sound can accept all three, while a system built on sound would accept none of them.
The Melody Fingerprint
Google’s own explanation uses a simple image: a song’s melody is like its fingerprint, with its own unique identity. When you hum into Search, machine learning models transform the audio into a number-based sequence representing the song’s melody, and that sequence is compared with thousands of songs to find potential matches in real time.
Building that sequence means throwing information away on purpose. The models are trained on humans singing, whistling or humming, as well as studio recordings, and the algorithms strip out the accompanying instruments and the voice’s timbre and tone. What is left is the tune: the rise and fall of pitch over time, without the sound that carried it.
That stripping is what makes the feature forgiving. If you listen to a song, you will recognise it whether it was sung, whistled or hummed, and Google’s models are built to do the same: they recognise the melody of the studio version and match it with a person’s hummed audio, regardless of how it was performed.
Embeddings: Teaching a Network What “Same Melody” Means
Under the fingerprint idea sits a neural network technique called embeddings. Instead of producing an intermediate representation of the melody, Google’s system produces an embedding of a melody directly from a spectrogram of the song: a numerical representation that places similar things close together.
Training works on pairs. The network is fed pairs of audio, a hummed or sung rendition alongside the recorded version, and learns to produce embeddings where pairs containing the same melody sit close to each other, even with different instrumental accompaniment and singing voices, while pairs containing different melodies sit far apart.
Finding the song is then a search problem: generate an embedding for the hum, and look for similar embeddings among reference recordings computed from popular music. The model must search among tens of millions of spectrogram segments from studio recordings, and it must focus on the dominant melody while ignoring background vocals, instruments, voice timbre, background noise and room reverberation.
The payoff of this design is a database that stays fresh. Because the system needs no hummed or MIDI version of each track, its catalogue can be refreshed constantly with embeddings of original recordings from around the world, including the latest releases, rather than waiting for someone to record a hummed version of each new song.
The Training Problem: Almost Nobody Hummed
The hardest part of building Hum to Search was not the model. It was the training data. Training required pairs of recorded and sung audio, and Google’s initial dataset consisted mostly of sung segments, with very few containing humming. To make the model more robust, the team augmented the audio during training, varying the pitch or tempo of the sung input at random.
That produced a model that worked well for people singing, but not for people humming or whistling. The fix was to synthesise humming. Using SPICE, a pitch extraction model developed by Google’s wider team as part of the FreddieMeter project, researchers extracted pitch values from existing audio and generated melodies consisting of discrete tones, creating simulated hummed training data from sung recordings.
The first version of that synthesis used a simple tone generator. Google later refined it by replacing the tone generator with a neural network that generates audio resembling an actual hummed or whistled tune. As a final step, the team mixed and matched audio samples, aligning clips from two different singers of the same melody with preliminary models, so the network saw additional pairs representing the same tune in different voices.
The same gap between a human rendering and the original shows up in images too, which our guide to how Google Lens and Ask Photos work covers for photos.
Triplet Loss and the Confidence Fix
Training the model started with a loss function called triplet loss, a standard technique across classification tasks including images and recorded music. Given a pair of audio for the same melody, triplet loss ignores certain training examples from a different melody: the ones that are already far away and therefore easy, and the ones that are anomalously close and therefore too hard for the model’s current state of learning.
Ignoring those examples wastes information, so Google added a second idea: a general notion of model confidence across a batch of examples. How sure is the model that all the data it has seen can be classified correctly, and has it seen examples that do not fit its current understanding? From that notion, the team added a loss that drives model confidence toward 100 percent across all areas of the embedding space.
The combination improved the model’s precision and recall, and with the training-data variations, augmentations and superpositions, it enabled the network deployed in Google Search to recognise sung or hummed melodies. Google describes the resulting system as reaching a high level of accuracy on a song database of more than half a million songs, continually updated, with room to grow to include more of the world’s melodies.
How to Get a Better Match
First, hum the most recognisable part, usually the chorus or the hook, rather than a verse or an intro. The hook is the part of the melody with the most distinctive shape, and it gives the model the most to match against.
Second, hum for the full 10 to 15 seconds if you can. A longer sample gives the model more melody to work with, and a two-second fragment of a common chord pattern can match dozens of songs.
Third, find a quiet moment. The model is built to ignore background noise and room reverberation, but a quiet room still gives it a cleaner melody line to convert into a sequence.
Fourth, if humming fails, try whistling or singing. The model was trained on all three, and a whistle’s clean pitch line or a sung version with the right rhythm can succeed where a hum did not. If the top match is wrong, check the other likely options and their match percentages before humming again.
What Happens After a Match
A match is not the end of the experience; it is the start of the song’s page. Selecting the best match lets you explore information on the song and artist, view accompanying music videos, and listen to the song on your favourite music app. You can also find the lyrics, read analysis, and check out other recordings of the song when available.
The match percentages matter here. They are the system’s own confidence, and they tell you how much to trust the top result. A high-percentage top match with a big gap to the second result is usually the song. A cluster of close percentages means the hum matched several melodies nearly equally, and the right answer may be second or third on the list, waiting for your ears to confirm it.
What Hum to Search Cannot Do Yet
The catalogue is the first limit. At the time of Google’s research post, Hum to Search covered more than half a million songs, far fewer than Sound Search’s 100 million-plus recorded catalogue. The hum catalogue has grown since, but an obscure track, a very new release, or a melody from outside the popular-music catalogue may simply not be in it yet.
The second limit is ambiguity. Some melodies are shared: the same tune can belong to more than one song, and a short or generic hum can match several candidates. In those cases the feature does what it is designed to do, showing the most likely options, and the final identification still needs your ears.
The third limit is the input itself. A hum that drifts badly off key, changes tempo halfway, or mashes two songs together may not contain a clean enough melody line for any model to match. The system is forgiving about pitch, but it cannot recover a melody that was never really there.
A fourth limit is versions. The system identifies the song, not necessarily the exact recording you heard. A melody covered by many artists, or a song with famous remixes and live versions, may return the song while leaving you to pick the right version from the recordings and listening options on the result.
Background
Hum to Search grew out of a decade of Google music recognition: Now Playing on Pixel in 2017, Sound Search in the Google app in 2018, and then the hum problem in 2020, solved by turning melodies into embeddings and training on synthesised humming. It remains a Search feature at heart: no lyrics, no artist name, no perfect pitch, just a tune and a fingerprint. Browse the subjects we cover on our topics index.
Sources: Google Blog: “Hum to search”; Google Research: “Machine learning behind Hum”.
0 Comments