kaaarim.com·Write-up·21 Sep 2026

Transcribing Egyptian Arabic meetings when half the vocabulary is English

I tested eight speech-to-text engines on Egyptian Arabic mixed with English. Google's general Gemini Flash models were the most accurate on a real recording, at 6–9% word error for about $0.06 an audio hour. Google's own purpose-built transcription model came second to last and cost the most. Every engine that lost, lost the same way: on the English words inside Arabic sentences.

The question came from an office in Cairo that wants its meetings transcribed. People there speak Egyptian Arabic and switch to English mid-sentence for anything technical, often a single word with the Arabic article glued to the front. Every vendor publishes an Arabic accuracy figure, and every one of them is measured on people reading Modern Standard Arabic aloud. Nobody in that office talks like that, so I measured it myself over two days.

Three things that matter more than the ranking

1. English handling decides everything. No engine lost because it was bad at Arabic. Whisper translates the English into Arabic. Speechmatics and Google's transcription model drop it. Speechmatics also transliterates it into Arabic letters. Evidence: three public datasets and one real recording
2. Google's purpose-built transcription model is the worst of its own lineup. It kept 32 of 66 English words where the general Flash models kept 63 to 67, and caught 1 of 9 key English terms against their 7 or 8. It costs five times as much as Flash-Lite. Evidence: one 2-minute recording (preliminary)
3. Published accuracy numbers say nothing about dialect. On read Modern Standard Arabic, Cohere's model scored 4.0% word error. On real conversational Egyptian with English, the same model scored 22%. Google publishes no Arabic-specific figure for its transcription model at all. Evidence: FLEURS vs two dialect datasets

The ranking, on a real recording

Two minutes recorded on a phone: one speaker, Egyptian Arabic with English, deliberately varying distance and volume. It's the only audio guaranteed to be in nobody's training data.

Read this with care. One recording, one speaker. The reference transcript was produced with Gemini 3.8 Flash, so its own 4.6% is self-agreement, not accuracy. 3.5 Flash and Flash-Lite are independent of it and agree with each other. Flash-Lite's 67 of 66 means it wrote more English than the reference has.

The recommendation is Flash-Lite on the paid tier: the cheapest accurate option, at around $4–5 a month for 80 hours of meetings. Cohere stays installed as a free fallback that never sends audio anywhere. At 16% it's usable.

The ranking, on public data

Gemini wasn't in this round; the four other engines were, on 28 minutes of real conversational Egyptian with 641 English words, from a small public dataset unlikely to be in any model's training data:

"Meaning kept" is the share of clips whose transcript still says what was said (method below). Local engines ran on a laptop RTX 4070 with 8 GB.

That isn't "free beats paid". Cohere's model is a purpose-built Arabic model from a major lab, trained for dialect and code-switching: a specialist beating generalists. And the domain matters. On Egyptian TV dialogue with almost no English, the order flips and Speechmatics wins clearly, keeping the meaning of 76% of clips against Cohere's 54%. On the real recording above, Speechmatics was also far closer to Cohere than on this dataset.

What going wrong looks like

All examples below are from the public datasets.

Google's transcription model failed in the same way as Speechmatics on the real recording: it dropped a whole English clause, one that three other engines all transcribed, and it did so despite Google listing Egyptian Arabic and mid-sentence code-switching as supported. A custom dictionary doesn't help either. Giving Speechmatics the 25 English terms that kept recurring moved it from 146 to 147 words. The words aren't being misheard. They're being dropped.

Scoring meaning, not spelling

Word error rate is the standard metric, and for Egyptian Arabic it misleads in two ways.

It counts spelling as error. Egyptian Arabic has no standard spelling, and no settled rule on whether "and" attaches to the next word. WER charges twice for that. On Egyptian TV audio, removing just that and one other spelling convention took Cohere from 31.4% to 25.5%, a fifth of the headline number for differences no reader would notice.

It can't see meaning. "user" written as اليوزر counts as an error, and so does a fluent mistranslation, equally.

So I also scored meaning: the reference and the transcript both go into one embedding space that spans Arabic and English (LaBSE), and I measure how close they land. I tested the judge before trusting it, against deliberately tricky pairs and against real clips with a known-good and a known-bad transcript. LaBSE separated them; the popular multilingual-e5 rated almost everything as similar, unrelated sentences included.

How I tried to fool myself, and caught it

Most of the work was ruling out results that looked good for the wrong reason.

• A model that had seen the answers. On the popular ArzEn dataset Cohere scored a suspicious 7.8%. It was reproducing the dataset's typed annotation markers, [HES] and [LAUGHTER], on 36% of clips against the reference's own 39%. Those are keystrokes with no sound, so it had read the dataset. Speechmatics produced them 0% of the time. The harness now checks for this on every run.
• A 100% that meant nothing. The English metric once reported "100% kept" on a dataset with one English word in 2,814. It now refuses to print a percentage below 25 words.
• A suspicion that turned out wrong. Speechmatics had been sent one long file while the local engines got pre-cut clips. That's a real difference, so I re-sent 100 clips as separate jobs. 39.8% became 40.1%, and the ranking held.
• A red flag I withdrew. On a second dataset Cohere also echoed a marker from the reference. But that marker is just how you attach the Arabic "the" to an English word, so any engine writing English in Latin script produces it. The engines that don't produced it 0% of the time. Not contamination.
• A success that was actually a failure. Google's transcription model returns its text in a different field from the chat models. Reading the usual one gives an empty string with HTTP 200 and a normal "stop", which looks exactly like the model heard silence, and is billed in full.
• Checking the ruler. The Egyptian TV dataset's own paper reports Whisper large-v3 at 48.95% word error. My harness measured 47.5% independently, so it measures what it claims to.
• The number to plan against. Cohere scores 10.4% on perfectly cut clips and 22% on speaker turns from the diarizer, which is how it will actually run. The worse number is the honest one.

What it costs, and what to record with

September 2026 prices. Gemini 3.8 Flash is on introductory pricing that doubles on 1 January 2027. Audio bills at 25 tokens a second, measured from the API's own usage report, not the 32 commonly quoted.

• Cut the silence. Billing is by duration, so compressing audio saves nothing, but a quarter to 40% of a meeting is silence, and cutting it cuts the bill by the same share.
• Don't use the free tier for other people's meetings. Google may use free-tier content to improve its products, and human reviewers may see it. The paid tier doesn't train on your data.
• Don't connect a conference speakerphone to a phone over Bluetooth. Bluetooth microphones fall back to phone-call quality, 8 or 16 kHz and heavily compressed, without telling you. That's worse than the phone's own mic. The same speakerphone over USB to a laptop in the room keeps its full quality.

Who said it

Transcripts are only half the job; the other half is names. Each person is enrolled once, with 30–60 seconds of them speaking alone, and every meeting after that matches voices against those samples. On the test recording, enrolling 20 seconds took the match from "unknown" (similarity 0.09) to the right name (0.93). Anyone never enrolled, such as a client or a freelancer, stays "unknown", which is the correct answer.

What a transcript at this accuracy is good for

Errors are random and themes repeat, so patterns across many meetings survive well, while any single specific fact should be treated as a lead to check against the audio. One exception: a name mistranscribed the same way every time doesn't wash out with volume. It becomes a consistent, wrong fact.

What's still unknown

• Whether Gemini 3.8 Flash beats Flash-Lite. Its only score is circular; settling it needs a hand-checked reference.
• Several people talking over each other. Everything real so far had one speaker.
• A proper meeting-room microphone. Everything real so far was a bare phone.
• The office itself. Rankings flipped between TV and conversation, and meetings are a third kind of audio, so the first weeks should be spot-checked, not assumed.

Method

Engines

Whisper large-v3, Cohere Transcribe Arabic 2B and Qwen3-ASR 1.7B locally; Speechmatics (Arabic–English pack) and four Gemini models through their APIs.

Datasets

Code-switched Egyptian (Nash-pAnDiTa/ASR_En_Ar_CodeSwitching, 28 min), Egyptian TV (Casablanca, 31 min), ArzEn interviews (26 min, used as a contamination example), FLEURS read speech as the easy baseline, and one private 2-minute recording that isn't published.

Pipeline

Diarize first, then transcribe each speaker turn, so every engine gets identical audio.

Metrics

Word and character error; English words written in Latin script; meaning similarity (LaBSE), where "kept" is ≥ 0.70.

Hardware

Laptop RTX 4070, 8 GB.

When

20–21 September 2026, for under a dollar of cloud API spend. Models move fast; re-test before relying on this.