Sukhraj Singh
All projects

Computer vision · 2026

Temporal-Consensus OCR

More reliable video headlines by reading across frames.

Research · Controlled benchmarkIndependent research revision

My contribution

I rebuilt the OCR pipeline, deterministic benchmark, evaluation, accessible report and four-page manuscript.

Evidence

Open repository, downloadable paper and a reproducible 50-clip benchmark with machine-readable results.

Scope & limitations

The benchmark is synthetic and uses a known single-line region; real broadcasts, languages and automatic localization remain unverified.

  • Python
  • RapidOCR
  • OpenCV
  • ONNX Runtime
  • Accessibility

The research, briefly.

This research asks whether lightweight temporal aggregation can recover a known headline region more accurately than single-frame OCR. The CPU-oriented pipeline compares raw, contrast-equalized and binarized hypotheses, filters them with confidence-aware rules, resolves static text with a weighted medoid and reconstructs scrolling text through fuzzy token overlap.

Controlled corpus
50 four-second clips
Mean character error
12.34% to 2.10%
Exact match
70.0%
Threshold success
40/40 positive clips
The problem & approach

The problem

Text in news lower-thirds and tickers may never appear in audio or captions, and a moving headline may not be fully visible in any single frame. I wanted a local, inspectable method that combines temporal evidence while keeping its evaluation and limits reproducible.

The approach

This research asks whether lightweight temporal aggregation can recover a known headline region more accurately than single-frame OCR. The CPU-oriented pipeline compares raw, contrast-equalized and binarized hypotheses, filters them with confidence-aware rules, resolves static text with a weighted medoid and reconstructs scrolling text through fuzzy token overlap.

How it works
  1. 01Video frames
  2. 02Normalized headline region
  3. 03Raw + CLAHE + Otsu hypotheses
  4. 04RapidOCR recognition
  5. 05Confidence-aware filtering
  6. 06Static consensus or ticker assembly
  7. 07JSON + accessible HTML
  • I created a resolution-independent sampling and headline-region pipeline for static and scrolling text.
  • I implemented quality-weighted consensus for static overlays and fuzzy chronological overlap for tickers.
  • I generated a deterministic 50-clip benchmark spanning two motion modes, four degradations and ten negative controls.
  • I compared center-frame, multiview and temporal conditions with character, word, exact-match and sequence-success measures.
  • I exported inspectable JSON, regenerated result tables and a high-contrast browser report with local text-to-speech controls.
Technical decisions & lessons

Temporal aggregation

A scrolling sentence may be structurally incomplete in every individual frame, so evidence must be assembled over time.

Local CPU inference

The pipeline remains inspectable and usable without sending video content to a cloud OCR boundary.

Synthetic benchmark

A fixed generator makes the mechanism and degradations reproducible while clearly limiting external-validity claims.

Accessible output

Visible text, keyboard-friendly controls and local speech synthesis make the recovered headline useful beyond a raw OCR string.

Challenges

  • Ticker assembly must join partial text without turning a plausible overlap into an incorrect word.
  • The supplied one-line region keeps the experiment focused but does not solve full-frame text detection or tracking.
  • A 100% threshold-success result on the controlled positives is not 100% transcription accuracy; exact match remains 70.0%.

What I learned

  • I learned to separate mechanism validation from claims about real-world readiness.
  • I learned that temporal evidence contributes the gain; preprocessing alone did not improve the center-frame baseline.
  • I learned to keep OCR candidates and evaluation artifacts visible enough for someone else to audit.

Verified benchmark result · evaluation

center-frame CER: 12.34%
temporal CER:      2.10%
exact match:       70.0%
threshold success: 40 / 40 positives