Domain-Specific ASR: Why General Models Fail in the Real World

I was there when the medical transcription system turned "epinephrine 0.3 milligrams" into "a pen a friend 3 milligrams" - a 10x dosing error. That wasn't a bug. That was a general-purpose ASR model doing exactly what it was trained to do. After 12 years building speech recognition systems, I've watched this scene play out dozens of times. The demo goes perfectly. Production fails catastrophically.

Updated February 2026: Added Domain Adaptation Gap section and ASR Evaluation Protocol.

The problem is that 85% accuracy on general speech means 15% errors on your domain vocabulary - and domain vocabulary is where errors actually matter. The model never saw "epinephrine" because it learned from podcasts, YouTube, and phone calls, not medical dictation. I've seen this failure mode dozens of times across healthcare, legal, manufacturing, and aviation.

The myth of "one model to rule them all" dies fast in production environments. This is another manifestation of the demo-to-production gap.

Why General Models Fail

Modern ASR systems are trained on massive datasets. Hundreds of thousands of hours of transcribed audio. This gives them impressive performance on general speech. The problem is what "general" means.

General speech is:

• Conversational topics: weather, sports, news, daily life
• Common vocabulary: the 10,000 most frequent words cover 90%+ of everyday speech
• Standard pronunciation: how words "should" sound
• Clear context: sentences that make grammatical sense

Domain speech is:

• Technical topics: procedures, specifications, protocols
• Specialized vocabulary: jargon, abbreviations, codes
• Non-standard pronunciation: how practitioners actually say things
• Implicit context: meaning that depends on domain knowledge

The gap between these two worlds is massive. More training data won't close it. The training data doesn't contain the domain vocabulary. Research on United-MedASR demonstrates that domain-specific approaches using specialized glossaries from sources like ICD-10 and FDA repositories can achieve sub-1% error rates - results general models simply cannot match.

The Domain Adaptation Gap

To a general ASR model like Whisper, the phrase "Turn to heading 240" and "Turn to wedding 240" are equally plausible. The model has no way to know that "heading" is vastly more likely in a maritime context. It learned from general speech, where weddings come up more often than navigation.

This is where domain-adapted models diverge from general-purpose ones. A model that understands maritime context knows that "wedding" essentially never appears in bridge communications. The domain knowledge changes what the model considers probable.

The gap between general and domain-adapted models widens as technical vocabulary increases. In casual conversation, general models perform well. But as the density of specialized terminology rises, general models don't degrade gracefully - they fall off a cliff. The same model that transcribes podcasts flawlessly becomes unreliable when faced with medical dictation or air traffic control.

Healthcare: Where Words Kill

Medical transcription is where domain-specific ASR matters most. It's also where general models fail most dangerously.

Drug names:

• "Hydroxyzine" becomes "hydro cuisine"
• "Epinephrine" becomes "a pen a friend"
• "Atorvastatin" becomes "a tour vast a tin"
• "Metoprolol" becomes "me topple all"

Dosages:

• "0.3 milligrams" could become "3 milligrams" (10x error)
• "Q6H" (every 6 hours) becomes "Q16H" or "cute sex age"
• "BID" (twice daily) becomes "bid" (like an auction)

Procedures and anatomy:

• "Cholecystectomy" becomes "Collie cyst ectomy"
• "Bilateral" becomes "buy lateral"
• "Subcutaneous" becomes "sub cute anus"

I wish these were hypothetical examples. They're real transcription errors from production medical ASR systems. As Slator's analysis shows, even the most advanced general-purpose models like Whisper produce word error rates in medical contexts that would be unacceptable in clinical practice.

The consequences: wrong medications, wrong doses, wrong procedures in medical records. Patient safety incidents. Malpractice exposure. Staff learn not to trust the system, negating its value. These errors compound when combined with speaker diarization failures. Now you have the wrong words attributed to the wrong person.

Legal: Precision Under Scrutiny

Legal transcription demands accuracy that general models can't provide.

Case citations:

• "28 USC 1332" (the federal diversity jurisdiction statute) becomes "28 you see 1332"
• "Miranda v. Arizona" becomes "Miranda v Arizona" (losing the period matters for citation format)
• "FRCP 12(b)(6)" becomes "FRC P12 B6"

Latin terms:

• "Res judicata" becomes "race you decada"
• "Pro bono" becomes "pro bone oh"
• "Prima facie" becomes "premium facie"

Procedural language:

• "Voir dire" becomes "for deer"
• "Habeas corpus" becomes "have he is corpus"
• "Amicus curiae" becomes "a Mikus curry eye"

In a deposition or court proceeding, these errors matter. The record must be accurate. Attorneys reviewing transcripts need to trust them. One garbled citation can require hours to reconstruct.

Manufacturing: The Jargon Jungle

Every factory has its own language. Part numbers, machine names, process steps. None of which appear in general training data.

Part specifications:

• "M8x25 hex head bolt" - every element is domain-specific
• "6061-T6 aluminum" - alloy designation
• "0.001 inch tolerance" - precision measurement

Machine names:

• "The Haas VF-2" (a CNC mill) becomes "the Haas VF two" or worse
• "Fanuc robot" becomes "phonetic robot"
• "Mazak lathe" becomes "my sack lathe"

Process terminology:

• "Anodize" becomes "analyze"
• "Chamfer" becomes "chamber"
• "Deburr" becomes "defer"

For quality control and compliance documentation, these errors create problems. ISO auditors don't accept transcripts with obvious errors. Manufacturing systems can't parse garbled part numbers.

Aviation: Where Miscommunication Is Fatal

Aviation has a language designed to be unambiguous over noisy radio. General ASR butchers it.

Phonetic alphabet:

• "Alpha Bravo Charlie" should stay exactly that, not "alpha brave charlie"
• "Niner" (how pilots say 9) becomes "minor" or "nicer"
• "Fife" (how pilots say 5) becomes "fife" (at least) or "five"

Callsigns and instructions:

• "Delta 1234 heavy" becomes "delta 1234 heavy" (missing the airline context)
• "Cleared ILS runway 27L" becomes "cleared ails runway 27 L"
• "Squawk 7500" (hijack code) becomes "squawk 7500" (missing the critical context)

Altitudes and headings:

• "Flight level three five zero" (35,000 feet) needs to be parsed correctly
• "Heading two seven zero" needs to stay exactly that

Aviation transcription errors have contributed to incidents. When the stakes are this high, "close enough" isn't enough.

ASR Stress Test: Domain Tongue Twisters

Copy these phrases and test any ASR system. If it fails more than 2, the model isn't ready for your domain:

Building Domain-Specific Models

The solution isn't to wait for general models to get better. Build or fine-tune models for your domain.

Step 1: Gather Domain Data

You need audio and transcripts from your actual environment. Not scripts. Not simulations. Real recordings.

• Volume: Minimum 100 hours for basic fine-tuning. 1,000+ hours for robust performance.
• Variety: Different speakers, conditions, topics within the domain.
• Quality: Accurate transcriptions, verified by domain experts.

This is the expensive part. Transcribing 100 hours of audio takes hundreds of person-hours. But it's the foundation everything else builds on.

Step 2: Build the Vocabulary

Create a comprehensive list of domain terms:

• Technical terms and their pronunciations
• Abbreviations and how they're spoken
• Proper nouns (product names, machine names, location names)
• Codes and their meanings

This vocabulary feeds into both language model training and acoustic model biasing.

Step 3: Fine-Tune or Train

Options depending on your resources:

Vocabulary boosting: Add domain terms to the decoding vocabulary of an existing model. Cheapest, least effective.

Language model adaptation: Fine-tune the language model component on domain text. Moderate cost, good results for vocabulary issues.

Acoustic model fine-tuning: Fine-tune the acoustic model on domain audio. Higher cost, addresses pronunciation and noise issues.

Full training: Train from scratch on domain data. Highest cost, best results for truly specialized domains.

Step 4: Continuous Improvement

Domain vocabulary evolves. New products launch. Processes change. Jargon shifts.

Build a feedback loop:

• Capture corrections from users
• Track error patterns
• Periodically retrain on new data
• Monitor accuracy metrics continuously

The ROI of Specialized Training

Custom ASR training costs money. Is it worth it?

The math:

A medical practice transcribes 1,000 patient encounters per month. With general ASR at 85% domain accuracy:

• 150 errors per 1,000 transcripts need correction
• At 5 minutes per correction = 12.5 hours of correction work
• At $50/hour for medical professional time = $625/month in corrections
• Plus risk exposure from uncaught errors

With domain-specific ASR at 97% accuracy:

• 30 errors per 1,000 transcripts
• 2.5 hours of correction work
• $125/month in corrections
• Reduced risk exposure

Savings: $500/month, $6,000/year.

Custom model development costs $50,000-$200,000 depending on complexity. Payback period: 1-3 years. Plus avoided liability.

For high-volume operations, the ROI is clear. For critical operations like healthcare and aviation, the risk reduction alone justifies the investment.

The Hybrid Approach

Most organizations don't need fully custom ASR. They need general ASR that handles domain vocabulary correctly.

Our approach at AMBIE:

1. Start with a strong base model: Modern architectures (Whisper, Conformer) trained on diverse data.
2. Add domain vocabulary biasing: Boost recognition of domain terms without full retraining.
3. Fine-tune on target audio: Adapt to the acoustic environment (noise, radio quality, accents).
4. Add post-processing rules: Domain-specific corrections and formatting.
5. Build feedback loops: Continuous learning from corrections.

This gives 90% of the benefit of full custom training at 30% of the cost. Just remember that accuracy metrics can be deceiving. Measure what matters for your use case, not just overall WER.

When "Good Enough" Isn't

Some applications tolerate errors. Meeting transcription, podcast indexing, casual note-taking. If you miss a word here and there, no one dies.

Some applications don't:

• Medical orders: A transcription error can harm a patient.
• Legal proceedings: The record must be accurate.
• Emergency dispatch: A misheard address costs minutes.
• Aviation: Clearances must be exact.
• Financial transactions: Numbers must be right.

For these use cases, general-purpose ASR isn't a starting point. It's a non-starter. Domain-specific training isn't an optimization. It's a requirement.

"The myth of "one model to rule them all" dies fast in production environments."