m/mGPUmanager
1
0
Fork 0
Code Issues 13 Pull Requests Actions Packages Projects Releases Wiki Activity

STT forces German on every utterance: -l de hardcoded + language never forwarded, so English voice is translated to German #9

New Issue
Open
opened 2026-07-25 09:30:31 +00:00 by mAi · 3 comments
mAi commented 2026-07-25 09:30:31 +00:00
Copy Link

Symptom

English voice captures through the PWA come back wrong. In the worst case whisper.cpp does not transcribe the English audio, it translates it into German — verified live with an English clip returning German text.

Path

PWA voice → OTTO_PWA_MVOICE_URL (which, despite the name, points at mGPUmanager mrock:8770/v1/stt) → mGPUmanager routes stt → whisper-server.service (whisper.cpp large-v3-turbo). mVoice is not in the STT path at all; it only serves TTS. (Full path write-up: m/mVoice docs/stt-production-path.md, from the #22 investigation.)

Root cause — two parts, both required

  1. whisper-server.service hardcodes -l de. Verified ExecStart on mRock:

    whisper-server -m .../ggml-large-v3-turbo.bin -l de --host 0.0.0.0 --port 8178 \
  --convert --tmp-dir /tmp --prompt "Dania, Matthias, Otto, Jule, Hogan Lovells"

  2. mGPUmanager only forwards a language when stt_lang != auto. The PWA always sends auto. So the per-request language is never set, the service falls back to its hardcoded -l de, and auto-detect is unreachable. stt_language in the response is therefore a constant, not a detection.

Under -l de with no per-request prompt, an English clip comes back translated into German rather than transcribed.

Verified fix (apollo, #22 investigation)

  • Drop -l de from the service and set -l auto explicitly. Note: whisper.cpp's -l default is en, so removing the flag is NOT the same as -l auto — set it explicitly.
  • In mGPUmanager, forward language=auto to whisper-server instead of suppressing it, and report the real detected_language from whisper.cpp's verbose_json.

No German regression: auto vs de output is byte-identical on the German reference clip.

Acceptance

  • An English voice clip transcribes as English (not translated to German), verified live.
  • A German voice clip is unchanged (no regression), verified live.
  • The response's detected_language reflects the actual detected language rather than a constant.

Provenance

Surfaced by apollo (mvoice/gitster) during m/mVoice #22 (reported symptom was English punctuation loss; the real defect is this). ExecStart and the stt_lang != auto forwarding both independently re-verified by mvoice/head before filing. Filed here because the code and config live in mGPUmanager, not mVoice. Left unassigned — mGPUmanager owns the priority call.

Related: m/mVoice #22, and the glossary-drift defect filed alongside this one.

## Symptom English voice captures through the PWA come back wrong. In the worst case whisper.cpp does not transcribe the English audio, it **translates it into German** — verified live with an English clip returning German text. ## Path PWA voice → `OTTO_PWA_MVOICE_URL` (which, despite the name, points at **mGPUmanager** `mrock:8770/v1/stt`) → mGPUmanager routes `stt` → **whisper-server.service** (whisper.cpp `large-v3-turbo`). mVoice is not in the STT path at all; it only serves TTS. (Full path write-up: `m/mVoice` `docs/stt-production-path.md`, from the #22 investigation.) ## Root cause — two parts, both required 1. **`whisper-server.service` hardcodes `-l de`.** Verified ExecStart on mRock: ``` whisper-server -m .../ggml-large-v3-turbo.bin -l de --host 0.0.0.0 --port 8178 \ --convert --tmp-dir /tmp --prompt "Dania, Matthias, Otto, Jule, Hogan Lovells" ``` 2. **mGPUmanager only forwards a language when `stt_lang != auto`.** The PWA always sends `auto`. So the per-request language is never set, the service falls back to its hardcoded `-l de`, and auto-detect is unreachable. `stt_language` in the response is therefore a constant, not a detection. Under `-l de` with no per-request prompt, an English clip comes back **translated** into German rather than transcribed. ## Verified fix (apollo, #22 investigation) - Drop `-l de` from the service and set **`-l auto`** explicitly. Note: whisper.cpp's `-l` default is `en`, so *removing* the flag is NOT the same as `-l auto` — set it explicitly. - In mGPUmanager, forward `language=auto` to whisper-server instead of suppressing it, and report the real `detected_language` from whisper.cpp's `verbose_json`. No German regression: `auto` vs `de` output is byte-identical on the German reference clip. ## Acceptance - An English voice clip transcribes as English (not translated to German), verified live. - A German voice clip is unchanged (no regression), verified live. - The response's `detected_language` reflects the actual detected language rather than a constant. ## Provenance Surfaced by apollo (mvoice/gitster) during `m/mVoice` #22 (reported symptom was English punctuation loss; the real defect is this). ExecStart and the `stt_lang != auto` forwarding both independently re-verified by mvoice/head before filing. Filed here because the code and config live in mGPUmanager, not mVoice. Left unassigned — mGPUmanager owns the priority call. Related: `m/mVoice` #22, and the glossary-drift defect filed alongside this one.
mAi commented 2026-07-29 10:01:46 +00:00
Author
Copy Link

Fixed, code pushed, live verification below

Branch mai/curie/mgpumanager-stt-truth-9.
Commits: 730fe90 · cbebade

Both root causes fixed

1. The service. whisper-server.service now runs -l auto, set explicitly. Confirmed against this build's own help text: -l LANG, --language LANG [en] spoken language ('auto' for auto-detect) — the default really is en, so dropping the flag would not have been the same thing. Installed and restarted on mRock at 11:53:30; the old unit is kept as ~/.config/systemd/user/whisper-server.service.bak-curie. The unit joins the repo as systemd/whisper-server.service so the next change is reviewable.

2. The broker. proxySTTWhisperCpp now forwards a language on every request, auto included, asks for response_format=verbose_json, and reports the language whisper detected. stt_language used to be the constant "de" — a claim, not a measurement. whisper names languages in English ("german"), so the adapter maps to ISO-639-1; an unmapped language passes through verbatim rather than being guessed at, because the field is a log tag for the PWA and never a control input.

Acceptance, measured

End to end through a staging broker (port 8771) against the real whisper-server on mRock, in the PWA's own dialect (stt_lang=auto, multipart audio):

Clip user_text stt_language
English Hello Matthias, this is an English test sentence about the weather in London today. en
German Hallo Otto, das ist ein deutscher Test Satz über das Wetter in Düsseldorf heute. de
  • English in, English out. No translation.
  • German unchanged. auto and de produce byte-identical text on the German reference clip, reproducing apollo's measurement, so drift there stays a real signal.
  • detected_language tracks the audio rather than the config.

The clips are espeak-ng synthetic, so this proves the mechanism, not transcription quality. The translation symptom itself needs m's real voice — an espeak English clip is clean enough that even forced language=de transcribed rather than translated it, while still reporting "language":"german". The forcing was demonstrably present; the clip was just too easy to break on it.

One more defect found while verifying — the punctuation symptom

whisper carries the initial prompt's punctuation style into the transcript. Same audio, same terms, only the prompt's final character differs:

prompt "…, Juliane Brandes"   →  " Hallo Otto, … in Düsseldorf heute\n"
prompt "…, Juliane Brandes."  →  " Hallo Otto, … in Düsseldorf heute.\n"

Reproducible, two runs each, live against large-v3-turbo. Since m/mVoice#22's reported symptom was punctuation loss, this is not cosmetic. The generated prompt, the static fallback and the service's launch --prompt now all end in a full stop.

That exposed a second one: whisper sometimes emits a sentence-final mark as its own segment, and joining the segments delivered Juliane . to the PWA, which renders the raw string. normalizeTranscript closes that gap now.

Tests

TestSTTReportsDetectedLanguage, TestSTTForwardsConcreteLanguage, TestSTTMissingLanguageDefaultsToAuto, TestSTTClosesGapBeforePunctuation, plus the reworked TestSTTWhisperCppAdapter. Each was run as a negative control with its fix disabled, and each failed — build, vet and the full suite are green with them in place.

Still open

One unit edit is outstanding and held: the launch --prompt gains its closing full stop, which needs another daemon-reload plus restart. Held at the head's request while mRock measurements run.

The broker binary deploy to mRock is queued behind the VRAM stream's deploy, per the shared-binary rule.

Glossary work is on #10. Design note including what the choice costs: docs/designs/issue-9-10-stt-language-and-lexicon.md.

## Fixed, code pushed, live verification below Branch `mai/curie/mgpumanager-stt-truth-9`. Commits: [730fe90](https://mgit.msbls.de/m/mGPUmanager/commit/730fe90) · [cbebade](https://mgit.msbls.de/m/mGPUmanager/commit/cbebade) ### Both root causes fixed **1. The service.** `whisper-server.service` now runs `-l auto`, set explicitly. Confirmed against this build's own help text: `-l LANG, --language LANG [en] spoken language ('auto' for auto-detect)` — the default really is `en`, so dropping the flag would not have been the same thing. Installed and restarted on mRock at 11:53:30; the old unit is kept as `~/.config/systemd/user/whisper-server.service.bak-curie`. The unit joins the repo as `systemd/whisper-server.service` so the next change is reviewable. **2. The broker.** `proxySTTWhisperCpp` now forwards a language on every request, `auto` included, asks for `response_format=verbose_json`, and reports the language whisper detected. `stt_language` used to be the constant `"de"` — a claim, not a measurement. whisper names languages in English (`"german"`), so the adapter maps to ISO-639-1; an unmapped language passes through verbatim rather than being guessed at, because the field is a log tag for the PWA and never a control input. ### Acceptance, measured End to end through a staging broker (port 8771) against the real `whisper-server` on mRock, in the PWA's own dialect (`stt_lang=auto`, multipart `audio`): | Clip | `user_text` | `stt_language` | |---|---|---| | English | `Hello Matthias, this is an English test sentence about the weather in London today.` | `en` | | German | `Hallo Otto, das ist ein deutscher Test Satz über das Wetter in Düsseldorf heute.` | `de` | - English in, English out. No translation. - German unchanged. `auto` and `de` produce byte-identical text on the German reference clip, reproducing apollo's measurement, so drift there stays a real signal. - `detected_language` tracks the audio rather than the config. The clips are espeak-ng synthetic, so this proves the **mechanism**, not transcription quality. The translation symptom itself needs m's real voice — an espeak English clip is clean enough that even forced `language=de` transcribed rather than translated it, while still reporting `"language":"german"`. The forcing was demonstrably present; the clip was just too easy to break on it. ### One more defect found while verifying — the punctuation symptom whisper carries the initial prompt's punctuation style into the transcript. Same audio, same terms, only the prompt's final character differs: ``` prompt "…, Juliane Brandes" → " Hallo Otto, … in Düsseldorf heute\n" prompt "…, Juliane Brandes." → " Hallo Otto, … in Düsseldorf heute.\n" ``` Reproducible, two runs each, live against `large-v3-turbo`. Since m/mVoice#22's reported symptom was punctuation loss, this is not cosmetic. The generated prompt, the static fallback and the service's launch `--prompt` now all end in a full stop. That exposed a second one: whisper sometimes emits a sentence-final mark as its own segment, and joining the segments delivered `Juliane .` to the PWA, which renders the raw string. `normalizeTranscript` closes that gap now. ### Tests `TestSTTReportsDetectedLanguage`, `TestSTTForwardsConcreteLanguage`, `TestSTTMissingLanguageDefaultsToAuto`, `TestSTTClosesGapBeforePunctuation`, plus the reworked `TestSTTWhisperCppAdapter`. Each was run as a negative control with its fix disabled, and each failed — build, vet and the full suite are green with them in place. ### Still open One unit edit is outstanding and held: the launch `--prompt` gains its closing full stop, which needs another `daemon-reload` plus restart. Held at the head's request while mRock measurements run. The broker binary deploy to mRock is queued behind the VRAM stream's deploy, per the shared-binary rule. Glossary work is on #10. Design note including what the choice costs: `docs/designs/issue-9-10-stt-language-and-lexicon.md`.
mAi commented 2026-07-29 10:07:12 +00:00
Author
Copy Link

Deployed to mRock and verified against production

On main 7b2a0ba. Broker binary rebuilt from that merged main and deployed 12:05.

Live, through the production broker on mrock:8770/v1/stt, in the PWA's own dialect:

English clip → {"user_text":"Hello Matthias, this is an English test sentence about the weather in London today.","stt_language":"en","stt_ms":308}
German clip  → {"user_text":"Hallo Otto, das ist ein deutscher Test Satz über das Wetter in Düsseldorf heute.","stt_language":"de","stt_ms":288}
  • English in, English out. No translation. ✅
  • German reference clip unchanged. ✅
  • stt_language tracks the audio, not the config. ✅
  • Sentence-final punctuation present in both.

whisper-server.service runs -l auto on mRock, confirmed from the live ExecStart. Both edits are backed up (.bak-curie, .bak-curie2), as is the previous broker binary (bin/mgpumanager.bak-curie).

After the restart: /healthz ok, /v1/status lists all four consumers healthy, POST /v1/gamemode 200 and released cleanly (ollama back up, /api/tags 200). mDock failover untouched and answering.

The host-side comfyui vram_resident_mib: 6000 hand-edit was preserved; the repo says 13000.

What this does not prove

The clips are espeak-ng synthetic. They show the mechanism — the language is forwarded, detection is honoured and reported. They cannot show the original symptom, because espeak English is clean enough that even forced language=de transcribed rather than translated it. m's own English voice capture is the last check.

## Deployed to mRock and verified against production On main [7b2a0ba](https://mgit.msbls.de/m/mGPUmanager/commit/7b2a0ba). Broker binary rebuilt from that merged main and deployed 12:05. **Live, through the production broker on `mrock:8770/v1/stt`, in the PWA's own dialect:** ``` English clip → {"user_text":"Hello Matthias, this is an English test sentence about the weather in London today.","stt_language":"en","stt_ms":308} German clip → {"user_text":"Hallo Otto, das ist ein deutscher Test Satz über das Wetter in Düsseldorf heute.","stt_language":"de","stt_ms":288} ``` - English in, English out. No translation. ✅ - German reference clip unchanged. ✅ - `stt_language` tracks the audio, not the config. ✅ - Sentence-final punctuation present in both. `whisper-server.service` runs `-l auto` on mRock, confirmed from the live `ExecStart`. Both edits are backed up (`.bak-curie`, `.bak-curie2`), as is the previous broker binary (`bin/mgpumanager.bak-curie`). After the restart: `/healthz` ok, `/v1/status` lists all four consumers healthy, `POST /v1/gamemode` 200 and released cleanly (ollama back up, `/api/tags` 200). mDock failover untouched and answering. The host-side `comfyui vram_resident_mib: 6000` hand-edit was preserved; the repo says 13000. ### What this does not prove The clips are espeak-ng synthetic. They show the mechanism — the language is forwarded, detection is honoured and reported. They cannot show the original symptom, because espeak English is clean enough that even forced `language=de` transcribed rather than translated it. m's own English voice capture is the last check.
mAi commented 2026-07-29 10:15:17 +00:00
Author
Copy Link

Correction: what production returns now

The transcripts I posted earlier were from espeak-ng clips. mRock has m's own voice on disk — ~/dev/mVoice/samples/profiles/otto/ref_de.wav and ref_en.wav, with the reference text in metadata.json. Real audio, known transcript. Re-running the acceptance against those found a defect the synthetic clips could not.

m's real German voice, through production, before the fix:

"Guten Morgen, mein Name ist Matthias, ich arbeite als Anw alt in Deutschland …"

Anwalt split in two. whisper follows its tokenizer convention and omits the leading space when a segment continues the previous word — " … als Anw" then "alt in …" — and normalizeTranscript joined every segment break with a space. The word boundary signal was in whisper's output the whole time; strings.Fields discarded it.

Not introduced by the language or glossary work. Checked against three prompts, two runs each: the break appears with the old 11-term prompt as well, and only disappears with no prompt at all. It has been in production since the #7 polish, on every request that carries a per-request prompt. espeak never produced a mid-word segment break, which is why three rounds of synthetic verification missed it.

Fixed in 19a4279, on main beabc09, deployed. A segment joins with a space only when it begins with whitespace.

Production now, m's own voice:

DE → {"user_text":"Guten Morgen, mein Name ist Matthias, ich arbeite als Anwalt in Deutschland und baue gerne an verschiedenen Projekten.","stt_language":"de","stt_ms":176}
EN → {"user_text":"Good morning. My name is Matthias. I work as a lawyer in Germany and enjoy building things with technology.","stt_language":"en","stt_ms":159}

Both verbatim against the reference text. #9's acceptance is now met on m's real speech rather than on synthetic audio: his English capture transcribes as English, not translated, and stt_language reports en.

/healthz, /v1/status and /v1/gamemode verified after the restart; game mode released cleanly.

The standing lesson, since this project keeps reaching for espeak: a synthetic voice can verify a mechanism, never a transcript. The reference recordings are on the box.

## Correction: what production returns now The transcripts I posted earlier were from espeak-ng clips. mRock has m's own voice on disk — `~/dev/mVoice/samples/profiles/otto/ref_de.wav` and `ref_en.wav`, with the reference text in `metadata.json`. Real audio, known transcript. Re-running the acceptance against those found a defect the synthetic clips could not. **m's real German voice, through production, before the fix:** ``` "Guten Morgen, mein Name ist Matthias, ich arbeite als Anw alt in Deutschland …" ``` `Anwalt` split in two. whisper follows its tokenizer convention and omits the leading space when a segment continues the previous word — `" … als Anw"` then `"alt in …"` — and `normalizeTranscript` joined every segment break with a space. The word boundary signal was in whisper's output the whole time; `strings.Fields` discarded it. Not introduced by the language or glossary work. Checked against three prompts, two runs each: the break appears with the old 11-term prompt as well, and only disappears with no prompt at all. It has been in production since the #7 polish, on every request that carries a per-request prompt. espeak never produced a mid-word segment break, which is why three rounds of synthetic verification missed it. **Fixed** in [19a4279](https://mgit.msbls.de/m/mGPUmanager/commit/19a4279), on main [beabc09](https://mgit.msbls.de/m/mGPUmanager/commit/beabc09), deployed. A segment joins with a space only when it begins with whitespace. **Production now, m's own voice:** ``` DE → {"user_text":"Guten Morgen, mein Name ist Matthias, ich arbeite als Anwalt in Deutschland und baue gerne an verschiedenen Projekten.","stt_language":"de","stt_ms":176} EN → {"user_text":"Good morning. My name is Matthias. I work as a lawyer in Germany and enjoy building things with technology.","stt_language":"en","stt_ms":159} ``` Both verbatim against the reference text. **#9's acceptance is now met on m's real speech rather than on synthetic audio**: his English capture transcribes as English, not translated, and `stt_language` reports `en`. `/healthz`, `/v1/status` and `/v1/gamemode` verified after the restart; game mode released cleanly. The standing lesson, since this project keeps reaching for espeak: a synthetic voice can verify a mechanism, never a transcript. The reference recordings are on the box.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
No items
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
m
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: m/mGPUmanager#9
No description provided.
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?