ASR-generated subtitles vs forced alignment: why script-first captions fail less

A mistake I keep seeing in subtitle tools is simple but expensive: someone already has an approved script, but the workflow still starts by transcribing the audio again.

I ran into this while working with scripted voiceovers. The script was already reviewed, but the subtitle tool still wanted to guess the words from audio.

That sounds reasonable at first. Most captioning tools are built around speech-to-text. Upload audio, get words, split them into captions, export an SRT or VTT file.

But scripted video is a different problem.

If the script is already approved, ASR should not be the source of truth. It can help with timing evidence. It can help detect mismatches. But it should not quietly rewrite the words the user already signed off on.

That is the distinction I have been thinking about while building a script-first subtitle workflow.

The common mistake: transcribing an approved script again

For a lot of videos, the script comes before the audio.

Course lessons. Product walkthroughs. YouTube voiceovers. Localization review. Client-approved ads. Studio narration. The words are not unknown. They were written first, reviewed, then recorded.

In that situation, a transcription-first subtitle flow does extra work and creates a new risk:

audio -> ASR transcript -> subtitle splitting -> export

Once the ASR transcript becomes the source of truth, every downstream step inherits its substitutions.

That means the final caption file can differ from the approved script in small but annoying ways:

product names change
API terms get normalized
numbers become words, or words become numbers
casing disappears
punctuation gets simplified
domain-specific phrases turn into more common phrases

None of those changes may look dramatic in a demo. In production, they matter because subtitle files travel. They go into YouTube, editing timelines, localization handoffs, accessibility checks, client review, and sometimes legal or compliance workflows.

The approved script should own the words. The audio should only provide timing evidence.

What ASR is good at

ASR is very useful. I do not think this is an "ASR bad" argument.

ASR is the right default when you do not know the words yet:

audio -> transcript

That is the normal speech-to-text problem. NVIDIA's glossary describes speech-to-text as converting spoken language into written text, which is exactly what you want for meetings, interviews, podcasts, unscripted videos, and rough notes.

ASR also helps in scripted workflows. It can provide acoustic evidence, approximate word timing, confidence signals, and mismatch hints.

The boundary I care about is narrower:

when the user provides a script, the system should not treat ASR output as permission to replace that script.

What forced alignment is actually doing

Forced alignment is not the opposite of ASR.

That matters. A lot of aligners still use ASR models or acoustic models internally. NVIDIA's NeMo Forced Aligner, for example, uses CTC-based ASR models to generate token-, word-, and segment-level timestamps, and it can work with user-provided reference text. The key difference is the goal: generating new text versus locating known text in the audio.

ASR tries to recover text from audio.

Forced alignment tries to locate provided text in audio.

A simplified script-first flow looks more like this:

approved script
-> text normalization
-> audio segmentation
-> forced alignment / timing evidence
-> mismatch detection
-> cue generation
-> cue validation
-> review issue surfacing
-> SRT / VTT export

The product decision is not just "which model do we use?"

The product decision is "who owns the words?"

In a script-first caption system, the answer should be boring and strict: the script owns the words.

Why source-of-truth matters for subtitles

There is a difference between a transcript and a subtitle asset.

A transcript can be approximate. A human can skim it, search it, or fix it later.

A subtitle asset has to be consumed by other systems. It has timestamps. It has formatting constraints. It has line breaks. It has cue boundaries. It may be uploaded directly into a video platform or handed to another person who assumes it is ready.

That makes source-of-truth mistakes more expensive.

If an ASR-generated subtitle changes "VTT" to "VT", or "webhook" to "web hook", the file might still look plausible. The problem is that a plausible wrong caption is harder to catch than an obvious failure.

I would rather surface uncertainty than silently make the text feel clean.

Uncertainty should become a review issue, not a silent edit.

A small failure example

Here is the kind of thing that looks minor until it gets exported.

Original script:

Deploy the VTT file after the webhook returns 200.

Possible ASR output:

Deploy the VTT file after the web hook returns two hundred.

That is not a terrible transcript. A human understands it.

But it changed the asset:

webhook became web hook
200 became two hundred
developer wording became more conversational

For some content, that is fine. For a technical tutorial, docs video, client-approved narration, or localization source file, it is not the same text anymore.

There is also the classic funny version:

Upload the SRT to YouTube Studio before 9:00.

becoming:

Upload the shirt to YouTube studio before nine.

That one is easier to laugh at, but the more realistic failure is usually not a total mishearing. It is a slow loss of exact wording.

Engineering problems in script-first subtitles

Script-first alignment sounds simple:

script + audio -> timestamps

In practice, there are several places where the system has to be conservative.

Text normalization cannot become rewriting

Some normalization is useful. You may need to compare punctuation-light text, lowercase forms, number variants, or tokenized words.

But the comparison form is not the export form.

The exported subtitle should preserve the approved script text unless the user explicitly edits it. Normalization should help matching, not become a hidden rewrite step.

Audio and script often disagree slightly

Voiceover is human.

People skip small words, add a phrase, read a number differently, or repeat a sentence after a mistake. Sometimes the script is a near-final draft, not the exact recording.

The system needs to decide what to do when the evidence is messy.

My preference is:

preserve the submitted script
flag the uncertain span
ask for review when needed

The tempting shortcut is to "fix" the subtitle text with the ASR transcript. That may look smoother in the UI, but it breaks the source-of-truth contract.

Long audio creates drift

Short clips are forgiving. A 45-second demo can have slightly rough timing and still feel okay.

Longer voiceovers are less forgiving. A small boundary mistake early in the file can make later cues feel off. Long files need sectioning, local checks, and final cue validation instead of one optimistic pass.

Word timestamps are raw material, not subtitle files

Word-level timing is useful, but it is not a finished SRT.

Subtitles need cue boundaries. They need readable line breaks. They need timestamps that players accept. They need constraints around duration and reading speed.

This is where a lot of "we have word timestamps" demos fall short. Word timestamps are ingredients. The subtitle file is the product.

Validation: turning uncertainty into review issues

I think the validator is where a subtitle system earns trust.

At minimum, I would want checks like these before calling an export ready:

start_time < end_time
no overlapping cues
timestamps within audio duration
minimum cue duration
maximum cue duration
max characters per line
max lines per cue
reading speed limit
preserve approved script text
detect skipped script spans
detect repeated script spans
flag low-confidence alignment spans
flag audio-script mismatch
valid SRT / VTT timestamp formatting

Some of these are structural. Some are readability checks. Some are source-fidelity checks.

The source-fidelity checks are the ones I care about most in a script-first workflow. If the script says webhook returns 200, the exported caption should not become web hook returns two hundred just because the audio model found that easier.

When confidence is low, the system should not pretend. It should show the user where the issue is and why it matters.

That review surface is part of the product, not a nice-to-have debug panel.

When forced alignment is not the right tool

Forced alignment is not always the right tool.

Use ASR when:

there is no script
the speaker is improvising
the audio contains unscripted Q&A
the script is only a rough outline
multiple speakers go off-script heavily

Use script-first alignment when:

the script is approved
the voiceover follows the script closely
exact wording matters
the output is a subtitle asset, not just a transcript

This distinction keeps the tool honest.

If the audio is truly unscripted, forcing it onto a reference text creates a different kind of bad result. If the script is authoritative, transcribing it again creates avoidable text pollution.

Why I am building this into TimedSubs

I am building this workflow into TimedSubs:

script + voiceover audio -> timed subtitle files with QA before export

The narrowness is intentional. I do not want it to be a generic transcription clone or a video editor. The useful part is the constraint: when a script is provided, the script stays authoritative, and the system uses audio to produce timing evidence and review signals.

Checklist for builders

If you are building captions or subtitle tooling, these are the questions I would ask:

Does the system know whether the user has an approved script?
If a script exists, does the final subtitle text preserve it?
Are ASR results treated as evidence, or as replacement text?
Can the system detect skipped, repeated, or mismatched script spans?
Do long files stay stable, or does timing drift over time?
Are cue-level export blockers checked before download?
Can the user see why a segment needs review?

The last one matters more than it sounds.

People can tolerate a system that says "I am not sure about this span." They lose trust in a system that silently changes approved words and calls the result ready.

Takeaway

ASR is good when you need to discover the words.

Forced alignment is useful when you already know the words and need timing.

For scripted video, course, voiceover, and localization workflows, that difference changes the whole product design. The caption system should not casually turn an approved script into a fresh machine transcript.