AI language identification over IPC
The accurate model was Python-only and the product was C++. Three approaches were explored; the one that shipped moved the boundary instead of trying to erase it.
- Accuracy
99.2%
Accuracy
- Shipped model size
138 MB
Shipped model size
Processing-time figures for the IPC architecture are not published here: the available measurements are not reconcilable against a single baseline.
My contribution
I evaluated the candidate models, selected and integrated the speech model into the language identification pipeline, and designed and implemented the IPC architecture connecting the C++ product to the Python inference process.
Context
Language identification determines which language is being spoken in an audio track. It runs inside a C++ analysis product, as one check among many, and the existing implementation was correct about 91.5% of the time on the evaluation corpus.
That number is good enough to be useful and not good enough to be trusted, which is the worst place for an automated check to sit.
The problem
The models that would fix the accuracy gap were Python. The product was C++, performance-sensitive, and shipped as a native application to customers who install it themselves.
So the real problem was never model selection. It was how a native C++ product uses a Python ecosystem without becoming a Python application.
Choosing a model by measuring, not by size
The first candidate was an ECAPA-based language identification model. It was evaluated and turned out not to be viable for this use, so the work moved to Whisper.
I evaluated Whisper across model sizes on a corpus of 981 files. The intuition — bigger model, better accuracy — did not survive contact with the data. Large-V2, at 2.9 GB, scored 98.47%. Base, at 138 MB, scored 99.18%: better accuracy at one twenty-first of the size.
That result is less surprising than it looks. Language identification is a much narrower task than transcription, and the largest model is optimized for the harder problem. Base shipped.
Files misidentified, out of 981
Lower is better · value shown with resulting accuracy
Shown as error counts rather than accuracy: on a 0–100% axis every bar would sit in the same place.
Accuracy on a 981-file corpus. The 138 MB model outperformed the 2.9 GB one on this task.
Internal model evaluation · 981-file corpus
The pipeline around the model
Audio is resampled to 16 kHz to match the model’s training data, then passed through voice activity detection so that only speech segments reach it. Silence and noise produce confident nonsense otherwise.
Each speech segment yields a detected language and a confidence score. Rather than taking a majority vote across segments, the pipeline computes the mean confidence per language across the file and selects the highest. A vote treats a marginal segment and a certain one as equal; averaging confidence does not.
Dialect detection is layered on top and deliberately conservative: it is consulted only when both the primary model and the dialect model agree on the underlying language, so a dialect result can refine a language decision but never overturn it.
On-disk model size
Model sizes evaluated. The one that shipped is the second smallest.
Model sizes evaluated
Three approaches to the language boundary
Accuracy improved. Processing time got worse: the new model was doing considerably more work than what it replaced. Three approaches were explored to get that back.
A wrapper approach — binding the Python library directly into the C++ process — was explored. So was reimplementing the inference path in C++. The approach that shipped did neither: it moved the boundary instead of trying to erase it.
The product runs a C++ client that talks to a separate Python server process over named pipes, with a structured message layer for requests and responses. The Python side runs an optimized inference implementation and handles both language identification and speech-to-text; the C++ side stays native, and neither process has to pretend to be the other.
The C++ product and the Python inference process each stay in their own runtime, connected by a structured message layer over named pipes.
The hard part
A boundary you can move but not remove
Embedding Python in the C++ process makes deployment, interpreter lifetime, and failure isolation into permanent problems: an inference crash takes the whole application with it. Reimplementing the model in C++ means owning a reimplementation of something that is actively maintained elsewhere, and re-verifying it every time it changes.
Process separation costs a serialisation boundary and a message layer, which is real work. But it buys a crash domain, an independent upgrade path for the model, and the ability to use the Python ecosystem as it actually exists. That trade was worth making, and the cost is bounded in a way the other two are not.
Measured on a 981-file evaluation corpus. The shipped configuration corrected 973 of 981 files, against 898 for the implementation it replaced.
- Accuracy
- 91.5% → 99.2% on the evaluation corpus
- 99.2%
- Shipped model size
- 138 MB, selected over a 2.9 GB alternative that scored lower
- 138 MB
- Language coverage
- Extended to additional languages, including Czech, Danish and Turkish
Processing-time figures for the IPC architecture are not published here: the available measurements are not reconcilable against a single baseline.
Engineering takeaway
Some boundaries can be moved but not removed. Choosing where to put one beats pretending it is not there.