Fuzzy-matching voice commands with edit distance

A 60-year-old algorithm that beats both exact matching and an ever-growing list of misspellings.

The problem

I was building a real-time voice agent — the kind that listens to a live phone call and talks back. Most of it is speech-to-speech, but I needed a few out-of-band control commands the operator could speak to steer the session: things like "pause" and "resume".

The plan was simple. Transcribe the caller audio, and when a turn equals a known command, act on it:

if transcript.lower() == "pause":
    pause()

It worked great in a quiet room. In production it fell apart.

Why exact matching failed

Speech-to-text is excellent on full sentences and terrible on short, context-free utterances. A one- or two-word command gives the model almost nothing to anchor on, so it guesses — and the guesses are wild. Here’s what a single spoken command actually transcribed as, across a handful of attempts:

"pause"   ->  "paz"
"pause"   ->  "paws"
"resume"  ->  "redume"
"resume"  ->  "Review."
"resume"  ->  Видео он дэмо.   (yes, it switched languages)

Exact matching catches none of these. The command is acoustically obvious to a human but lexically all over the place.

The trap: a variant list

My first instinct was a lookup table of known spellings:

PAUSE = {"pause", "paz", "paws", "pos"}
if word in PAUSE:
    pause()

This is a treadmill. Every demo surfaced a new misspelling — pawz, posse, pas — and I was forever appending to the list. It only ever caught mistakes I'd already seen, never the next one.

Enter edit distance

The right tool is Levenshtein edit distance: the minimum number of single-character insertions, deletions, or substitutions to turn one string into another.

• edit_distance("paz", "pause") → small
• edit_distance("redume", "resume") → 1 (one substitution)
• edit_distance("review", "resume") → small

Instead of asking “is this word exactly a command,” I ask “is this word close enough to a command”:

is_pause  = any(edit_distance(w, "pause")  <= 2 for w in words)
is_resume = any(edit_distance(w, "resume") <= 2 for w in words)

One threshold replaces an ever-growing list, and it generalizes to misspellings I’ve never seen — because it measures similarity, not membership.

The whole algorithm is ~15 lines, no dependencies. The classic single-row dynamic-programming version:

def edit_distance(a, b):
    m, n = len(a), len(b)
    if not m:
        return n
    if not n:
        return m
    row = list(range(n + 1))
    for i in range(1, m + 1):
        prev_diag = row[0]
        row[0] = i
        for j in range(1, n + 1):
            tmp = row[j]
            row[j] = min(
                row[j] + 1,        # deletion
                row[j - 1] + 1,    # insertion
                prev_diag + (0 if a[i - 1] == b[j - 1] else 1),  # substitution
            )
            prev_diag = tmp
    return row[n]

Two small refinements that mattered

1. Pin the transcription language. The transcriber auto-detected per utterance and occasionally swung to Russian/Korean on a short command. Forcing it to one language killed the wrong-alphabet outputs that no edit-distance threshold could rescue.
2. Require a short, command-shaped turn. "pause" is also a normal English word. I only treat a turn as a command when it's a few words long, so a casual mid-sentence "pause" doesn't trip it.

The takeaway

Edit distance isn’t fancy — it’s a 60-year-old algorithm you can write from memory — but it’s the right shape for this problem: matching noisy input against a small, fixed vocabulary. When exact comparison is too strict and a hardcoded list is a maintenance treadmill, “close enough, by a measurable amount” is often exactly what you want.

Reach for it whenever you’re comparing human-or-machine-fuzzed strings to known targets: voice commands, typo-tolerant search, deduping near-identical records. And know its boundary — for phonetic similarity specifically, Soundex/Metaphone can beat it, and for ranking long strings, token-based similarity may fit better. But for “did they roughly say this command,” edit distance is hard to beat for the line count.