m/mGPUmanager
1
0
Fork 0
Code Issues 13 Pull Requests Actions Packages Projects Releases Wiki Activity

STT big-model via GPU manager: mVoice eager-loads at startup + PWA bypasses the broker → big model OOM-crashes on a full GPU #7

New Issue
Open
opened 2026-07-14 15:29:30 +00:00 by mAi · 3 comments
mAi commented 2026-07-14 15:29:30 +00:00
Copy Link

Problem (m wants mRock STT on the big model; naive swap failed live 2026-07-14)

m's STT quality is poor because the otto-PWA uses mVoice's small whisper model. Two coupled root causes make it impossible to just bump the model:

  1. The PWA bypasses the mGPUmanager broker. otto-pwa hits mVoice directly (OTTO_PWA_MVOICE_URL=https://mrock.horse-ayu.ts.net:8765/api/transcribe → :8766), never going through the broker /v1/stt (mrock:8770). So the manager never allocates GPU / evicts FLUX for STT.
  2. mVoice EAGER-loads its whisper model at STARTUP, unmanaged. mVoice/server.py loads the model during app startup (not lazily), directly on cuda:0, without acquiring a GPU lease from the manager.

Proven live: I bumped mvoice.service --whisper-model small → turbo and restarted. With FLUX (comfyui ~7GB) + Ollama (~4.7GB) resident (GPU 15.9/16GB used, ~50MiB free), turbo OOM'd at startup → torch.OutOfMemoryError: CUDA out of memory → crash-loop (exit 3) → STT down. Even reverting to small crash-looped (GPU still full) until I manually POSTed comfyui /api/free to reclaim ~5.8GB, after which small loaded and STT recovered. Rolled back to small; mVoice healthy again (loaded:true). Unit backup left at ~/.config/systemd/user/mvoice.service.bak-otto-stt-* on mRock.

So the big model literally cannot load while FLUX/Ollama fill the GPU unless the manager evicts them first at load time — exactly m's 'consider the GPU manager' point.

The fix (through the GPU manager — not a config swap)

  1. mVoice: make the whisper model load LAZY + manager-gated, not eager-at-startup. Load on first /api/transcribe (which arrives via the broker after eviction), OR have mVoice acquire a GPU lease from the manager before loading. Startup must NOT eager-load onto a full GPU. (m/mVoice/server.py.)
  2. Route the PWA at the broker: OTTO_PWA_MVOICE_URL → the mGPUmanager /v1/stt (mrock:8770, plain HTTP over tailnet) instead of mVoice direct. Then a voice request goes: PWA → broker → manager evicts comfyui/FLUX (mvoice priority 3 > comfyui 1) + ensures mvoice loaded → proxies to /api/transcribe. Response shape unchanged (broker proxies), so the PWA client + mVoice lexicon/German-prompt biasing keep working. (m/mAi deploy env — otto-pwa compose on mRiver Dokploy, composeId uPhRjsQL6NRcGm6tpDKix.)
  3. Bump mVoice to the big model (turbo = large-v3-turbo, what the whisper.cpp :8178 already uses and m remembers as good; ~1.6GB) once 1+2 guarantee GPU room at load. Update mgpumanager/config/consumers.yaml mvoice.vram_resident_mib to the real turbo footprint (measure; likely ~2500-3500) so the scheduler math is correct.
  4. Verify the broker's lease/eviction path (/v1/lease acquire → evict → ensureLoaded → proxy) actually frees enough for turbo, and that FLUX reloads cleanly after STT.

Acceptance

  • otto-PWA voice → broker → manager evicts FLUX → mVoice loads turbo → good German transcription with the name-lexicon biasing.
  • mVoice NEVER OOM-crashes at startup or on load, even with FLUX + Ollama resident (lazy/manager-gated load).
  • After an STT request, FLUX/comfyui reloads for image-gen (no permanent eviction).
  • Latency acceptable for live voice (first STT after a FLUX-evict pays the eviction cost; subsequent ones are fast).

Repos / coordination

Primary: m/mGPUmanager (broker lease/ensureLoaded + consumers.yaml) + m/mVoice (lazy/manager-gated model load + model flag). Plus m/mAi one-liner (otto-pwa MVOICE_URL → broker) deployed via mRiver Dokploy. Cross-repo — coordinate so all three land together (a half-applied change re-breaks STT). Context: memory 'otto/diagnosis PWA STT bypasses mGPUmanager' + 'otto/decision STT mRock primary'. mRock gotchas: fish login shell (ssh mrock bash -s), grep=rg (use /usr/bin/grep), auto-suspends 01-08h, macOS-style paths on mBreeze but mRock is Linux.

## Problem (m wants mRock STT on the big model; naive swap failed live 2026-07-14) m's STT quality is poor because the otto-PWA uses mVoice's **small** whisper model. Two coupled root causes make it impossible to just bump the model: 1. **The PWA bypasses the mGPUmanager broker.** otto-pwa hits mVoice directly (`OTTO_PWA_MVOICE_URL=https://mrock.horse-ayu.ts.net:8765/api/transcribe` → :8766), never going through the broker `/v1/stt` (mrock:8770). So the manager never allocates GPU / evicts FLUX for STT. 2. **mVoice EAGER-loads its whisper model at STARTUP, unmanaged.** `mVoice/server.py` loads the model during app startup (not lazily), directly on `cuda:0`, without acquiring a GPU lease from the manager. **Proven live:** I bumped `mvoice.service --whisper-model small → turbo` and restarted. With FLUX (comfyui ~7GB) + Ollama (~4.7GB) resident (GPU 15.9/16GB used, ~50MiB free), turbo OOM'd at startup → `torch.OutOfMemoryError: CUDA out of memory` → **crash-loop (exit 3)** → STT down. Even reverting to *small* crash-looped (GPU still full) until I manually POSTed comfyui `/api/free` to reclaim ~5.8GB, after which small loaded and STT recovered. Rolled back to small; mVoice healthy again (`loaded:true`). Unit backup left at `~/.config/systemd/user/mvoice.service.bak-otto-stt-*` on mRock. So the big model literally cannot load while FLUX/Ollama fill the GPU unless the **manager evicts them first at load time** — exactly m's 'consider the GPU manager' point. ## The fix (through the GPU manager — not a config swap) 1. **mVoice: make the whisper model load LAZY + manager-gated**, not eager-at-startup. Load on first `/api/transcribe` (which arrives via the broker after eviction), OR have mVoice acquire a GPU lease from the manager before loading. Startup must NOT eager-load onto a full GPU. (`m/mVoice/server.py`.) 2. **Route the PWA at the broker**: `OTTO_PWA_MVOICE_URL` → the mGPUmanager `/v1/stt` (mrock:8770, plain HTTP over tailnet) instead of mVoice direct. Then a voice request goes: PWA → broker → manager evicts comfyui/FLUX (mvoice priority 3 > comfyui 1) + ensures mvoice loaded → proxies to `/api/transcribe`. Response shape unchanged (broker proxies), so the PWA client + mVoice lexicon/German-prompt biasing keep working. (`m/mAi` deploy env — otto-pwa compose on mRiver Dokploy, composeId uPhRjsQL6NRcGm6tpDKix.) 3. **Bump mVoice to the big model** (`turbo` = large-v3-turbo, what the whisper.cpp :8178 already uses and m remembers as good; ~1.6GB) once 1+2 guarantee GPU room at load. Update `mgpumanager/config/consumers.yaml` `mvoice.vram_resident_mib` to the real turbo footprint (measure; likely ~2500-3500) so the scheduler math is correct. 4. **Verify** the broker's lease/eviction path (`/v1/lease` acquire → evict → ensureLoaded → proxy) actually frees enough for turbo, and that FLUX reloads cleanly after STT. ## Acceptance - otto-PWA voice → broker → manager evicts FLUX → mVoice loads **turbo** → good German transcription with the name-lexicon biasing. - mVoice NEVER OOM-crashes at startup or on load, even with FLUX + Ollama resident (lazy/manager-gated load). - After an STT request, FLUX/comfyui reloads for image-gen (no permanent eviction). - Latency acceptable for live voice (first STT after a FLUX-evict pays the eviction cost; subsequent ones are fast). ## Repos / coordination Primary: `m/mGPUmanager` (broker lease/ensureLoaded + consumers.yaml) + `m/mVoice` (lazy/manager-gated model load + model flag). Plus `m/mAi` one-liner (otto-pwa MVOICE_URL → broker) deployed via mRiver Dokploy. Cross-repo — coordinate so all three land together (a half-applied change re-breaks STT). Context: memory 'otto/diagnosis PWA STT bypasses mGPUmanager' + 'otto/decision STT mRock primary'. mRock gotchas: fish login shell (ssh mrock bash -s), grep=rg (use /usr/bin/grep), auto-suspends 01-08h, macOS-style paths on mBreeze but mRock is Linux.
mAi commented 2026-07-14 17:34:53 +00:00
Author
Copy Link

Interim shipped: otto-PWA STT now runs on the resident big model (large-v3-turbo)

m's PWA voice messages were garbling because STT ran on mVoice's small whisper model. Safe interim (zero OOM — routes to the already-resident whisper-server big model, does not eager-load a 2nd large model into mVoice; that CUDA-OOM crash-loop is the full #7, still separate).

Commit: 1e9d7c2 (branch mai/hopper/stt-bigmodel)

Why it wasn't just a config reroute

A verbatim broker proxy would have failed: whisper.cpp /inference requires multipart field file (the PWA sends audio → Invalid request 400) and returns {"text":...} (the PWA client requires {"user_text":...}). So the broker gained a small config-discriminated STT adapter (stt_adapter: whisper_cpp) that translates the mVoice dialect ↔ whisper.cpp both ways. whisper-server's --convert transcodes the PWA's WebM/Opus itself.

Changes

  • config/consumers.yaml: routing.stt mvoice → whisper-server; stt_adapter: whisper_cpp on whisper-server (validated in config.Load).
  • broker server: /v1/stt with an adapter consumer runs proxySTTWhisperCpp (audio→file rename, {text}→{user_text,stt_language,stt_ms}); added GET /api/health alias for the voice client's failover probe.
  • otto-pwa compose uPhRjsQL6NRcGm6tpDKix: OTTO_PWA_MVOICE_URL 8765/api/transcribe → 8770/v1/stt (only that line changed), redeployed.
  • tests: adapter dialect translation + health alias. go test -race ./... green.

Live verification (mRock)

  • Broker /v1/stt (multipart audio, stt_lang=auto): WAV → {"user_text":"Hallo Otto, hier spricht Matthias, das ist ein Test","stt_language":"de","stt_ms":93}; WebM/Opus (PWA's real format) → same transcript incl. punctuation, 121ms.
  • True tailnet path mRiver → http://mrock.horse-ayu.ts.net:8770/v1/stt → transcript, 133ms.
  • otto-pwa container restarted, healthy, running with the new URL.

Big model is large-v3-turbo with a German name-prompt (Dania, Matthias, Otto, Jule, Hogan Lovells) — a clear quality win over the small model. Pending m's PWA voice re-test for final acceptance.

Known follow-ups (not this task)

  • Full #7 (mVoice lazy/manager-gated load + turbo with the German lexicon biasing) remains the proper fix.
  • Minor interaction with the whisper-eviction bug (#8): after a comfyui image lease marks whisper-server "unloaded" (without stopping it), a broker STT can trigger one spurious eviction — benign (STT still transcribes), resolved once #8 lands.
## Interim shipped: otto-PWA STT now runs on the resident big model (large-v3-turbo) m's PWA voice messages were garbling because STT ran on mVoice's **small** whisper model. Safe interim (zero OOM — routes to the **already-resident** whisper-server big model, does **not** eager-load a 2nd large model into mVoice; that CUDA-OOM crash-loop is the full #7, still separate). **Commit:** [`1e9d7c2`](https://mgit.msbls.de/m/mGPUmanager/commit/1e9d7c2f1219b4818b625d3b90cc3f43933e08af) (branch `mai/hopper/stt-bigmodel`) ### Why it wasn't just a config reroute A verbatim broker proxy would have failed: whisper.cpp `/inference` requires multipart field **`file`** (the PWA sends **`audio`** → `Invalid request` 400) and returns **`{"text":...}`** (the PWA client requires **`{"user_text":...}`**). So the broker gained a small **config-discriminated STT adapter** (`stt_adapter: whisper_cpp`) that translates the mVoice dialect ↔ whisper.cpp both ways. whisper-server's `--convert` transcodes the PWA's WebM/Opus itself. ### Changes - `config/consumers.yaml`: `routing.stt` mvoice → whisper-server; `stt_adapter: whisper_cpp` on whisper-server (validated in `config.Load`). - broker `server`: `/v1/stt` with an adapter consumer runs `proxySTTWhisperCpp` (audio→file rename, `{text}`→`{user_text,stt_language,stt_ms}`); added `GET /api/health` alias for the voice client's failover probe. - otto-pwa compose `uPhRjsQL6NRcGm6tpDKix`: `OTTO_PWA_MVOICE_URL` 8765/api/transcribe → **8770/v1/stt** (only that line changed), redeployed. - tests: adapter dialect translation + health alias. `go test -race ./...` green. ### Live verification (mRock) - Broker `/v1/stt` (multipart `audio`, `stt_lang=auto`): **WAV** → `{"user_text":"Hallo Otto, hier spricht Matthias, das ist ein Test","stt_language":"de","stt_ms":93}`; **WebM/Opus** (PWA's real format) → same transcript incl. punctuation, 121ms. - True tailnet path mRiver → `http://mrock.horse-ayu.ts.net:8770/v1/stt` → transcript, 133ms. - otto-pwa container restarted, healthy, running with the new URL. Big model is `large-v3-turbo` with a German name-prompt (`Dania, Matthias, Otto, Jule, Hogan Lovells`) — a clear quality win over the small model. **Pending m's PWA voice re-test** for final acceptance. ### Known follow-ups (not this task) - **Full #7** (mVoice lazy/manager-gated load + turbo *with* the German lexicon biasing) remains the proper fix. - Minor interaction with the whisper-eviction bug (#8): after a comfyui image lease marks whisper-server "unloaded" (without stopping it), a broker STT can trigger one spurious eviction — benign (STT still transcribes), resolved once #8 lands.
mAi commented 2026-07-14 19:17:00 +00:00
Author
Copy Link

STT polish (both live-smoked on mRock)

Commit: 82fec70 (branch mai/hopper/stt-nameprompt)

1. Proper-noun name-prompt — mRock no longer → MROC

New stt_prompt config on the whisper-server consumer, forwarded as whisper.cpp's per-request prompt. Verified live that this overrides the service's static --prompt, so the configured value keeps the person names and adds m's infra hostnames: Dania, Matthias, Otto, Jule, Hogan Lovells, mRock, mRiver, mBreeze, mDock, mLake, mGPUmanager.

Live smoke (identical audio):

transcript of the infra names
RAW whisper :8178 (person-names prompt only) M-Rock … M-River … M-WC
via broker /v1/stt (full prompt) mRock … mRiver … mBre eze

mRock ✓ and mRiver ✓ now correct; mBreeze got a tiny space artifact from espeak's synthetic test voice — m's real voice should be clean.

2. Single-line transcript — no more visible line breaks in chat

whisper.cpp returns segment-delimited text with \n between segments; the adapter's {text}→{user_text} step now runs normalizeTranscript() (collapse \n + whitespace runs to single spaces, trim) instead of only trimming the ends.

Live smoke (3-sentence sample):

  • RAW whisper: "…zweite Nass. Zum\n Schluss… dritter Nass, zur Sicherheit.\n" (embedded \n)
  • via broker /v1/stt: single line, no breaks.

Deploy & tests

Binary-copy to mRock (comfyui vram_resident_mib: 11000 preserved; game-mode 200, healthz ok). Tests: prompt-forwarding assertion + TestSTTTranscriptSingleLine; go test -race ./... green.

Left the Satz→Nass / Klappt→Blatt acoustic slips alone — a prompt can't fix those; large-v3-turbo without mVoice's biasing does that occasionally, covered by full-#7.

## STT polish (both live-smoked on mRock) **Commit:** [`82fec70`](https://mgit.msbls.de/m/mGPUmanager/commit/82fec70e9d1b507882be741e358b52a8a38fe36a) (branch `mai/hopper/stt-nameprompt`) ### 1. Proper-noun name-prompt — `mRock` no longer → `MROC` New `stt_prompt` config on the whisper-server consumer, forwarded as whisper.cpp's per-request `prompt`. Verified live that this **overrides** the service's static `--prompt`, so the configured value keeps the person names *and* adds m's infra hostnames: `Dania, Matthias, Otto, Jule, Hogan Lovells, mRock, mRiver, mBreeze, mDock, mLake, mGPUmanager`. Live smoke (identical audio): | | transcript of the infra names | |---|---| | RAW whisper `:8178` (person-names prompt only) | `M-Rock … M-River … M-WC` | | via broker `/v1/stt` (full prompt) | **`mRock … mRiver … mBre eze`** | `mRock` ✓ and `mRiver` ✓ now correct; `mBreeze` got a tiny space artifact from espeak's *synthetic* test voice — m's real voice should be clean. ### 2. Single-line transcript — no more visible line breaks in chat whisper.cpp returns segment-delimited text with `\n` between segments; the adapter's `{text}`→`{user_text}` step now runs `normalizeTranscript()` (collapse `\n` + whitespace runs to single spaces, trim) instead of only trimming the ends. Live smoke (3-sentence sample): - RAW whisper: `"…zweite Nass. Zum\n Schluss… dritter Nass, zur Sicherheit.\n"` (embedded `\n`) - via broker `/v1/stt`: single line, no breaks. ### Deploy & tests Binary-copy to mRock (comfyui `vram_resident_mib: 11000` preserved; game-mode 200, healthz ok). Tests: prompt-forwarding assertion + `TestSTTTranscriptSingleLine`; `go test -race ./...` green. Left the `Satz`→`Nass` / `Klappt`→`Blatt` acoustic slips alone — a prompt can't fix those; large-v3-turbo without mVoice's biasing does that occasionally, covered by full-#7.
mAi commented 2026-07-29 10:15:53 +00:00
Author
Copy Link

Reassessment after #9 and #10: recommend close as superseded

Asked to judge whether #7-full is still worth building. Evidence below, then the recommendation.

Every structural item of this issue is already done — including the one that looked hardest

# This issue's fix State
1 mVoice: whisper load LAZY, not eager-at-startup Done. server.py calls _ensure_ready() inside the request handlers, and /api/admin/unload sets the model back to None. Live health on mRock right now: "loaded":false, "gpu_resident_mib":0. mVoice no longer eager-loads onto a full GPU, so the OOM crash-loop this issue was written around cannot recur.
2 Route the PWA at the broker Done, OTTO_PWA_MVOICE_URL → mrock:8770/v1/stt.
3 Big model + name-lexicon biasing Done, by a different route. The broker reaches large-v3-turbo on the already-resident whisper-server. Since #10 the lexicon is read from mVoice's own lexicon.yaml — the same terms, plus the same alias canonicalization mVoice applies.
4 Verify eviction frees enough for turbo; FLUX reloads Moot. No eviction happens: whisper-server holds ~2050 MiB and coexists with comfyui and ollama.

Acceptance, measured today against production with m's own reference recordings, not synthetic audio:

DE → "Guten Morgen, mein Name ist Matthias, ich arbeite als Anwalt in Deutschland und baue gerne an verschiedenen Projekten."  de, 176 ms
EN → "Good morning. My name is Matthias. I work as a lawyer in Germany and enjoy building things with technology."              en, 159 ms

Verbatim against the reference text, in both languages.

What #7-full would add if built now

A second copy of the same model. whisper-server already holds large-v3-turbo; item 3 would load large-v3-turbo into mVoice as well, on the GPU that is the contended resource. To make that safe it needs the lease-gated load, the eviction round trip and the FLUX-reload verification in items 1 and 4 — engineering whose entire purpose is to survive loading a model the broker can already query in 160 ms without evicting anything.

It is also slower by construction. The interim path costs one HTTP hop to a resident model. The #7-full path costs an eviction round trip on the first request whenever FLUX is resident — measured at 12.5 s for a comfyui lease during #5.

The acoustic slips — they are not #7-full's to fix, and they do not reproduce

The slips recorded against full-#7 (Klappt heard as Blatt, Satz as Nass) came from espeak-ng clips. Two findings:

They do not reproduce. Through production today:

espeak "Das klappt gut. Ein ganzer Satz zum Testen der Erkennung."
→ "Das klappt gut. Ein ganzer Satz zum Testen der Erkennung."

Verbatim. klappt is klappt, Satz is Satz.

And #7-full contains nothing that could have fixed them. Its scope is the same model (large-v3-turbo) with the same lexicon, on a different runtime. Acoustic decoding is not in it. Whatever those slips were — and the espeak artifacts observed since, mBrian heard as "im Bayern", suggest the synthesizer rather than the model — they were never evidence for this issue.

That distinction was the question, and it decides it: there is no remaining defect that #7-full's scope addresses.

Two things that are genuinely still open, and are not this issue

  1. mVoice's own STT still runs small. mvoice.service launches --whisper-model small, and mVoice's /api/chat, /api/respond and voice_loop.py transcribe with it. That is no longer on the PWA path, so it is a small, self-contained model-flag change in m/mVoice — and since mVoice already loads lazily, it no longer carries this issue's OOM risk. Worth its own issue if m still uses those surfaces; nothing here needs mGPUmanager.
  2. The mDock failover is degraded by design — whisper base on CPU, no lexicon. Different box, no GPU contention, unaffected by anything in #7-full.

Recommendation

Close as superseded (m closes issues, not us). Every acceptance bullet of #7 is met on the production path, verified on m's real voice; the one unbuilt item would add VRAM pressure and latency to reach a model the broker already reaches. If m wants mVoice's own STT on a bigger model, that is a one-line change in m/mVoice and deserves a fresh issue with its own justification, not this one's OOM framing — which no longer applies, because mVoice already loads lazily.

## Reassessment after #9 and #10: recommend **close as superseded** Asked to judge whether #7-full is still worth building. Evidence below, then the recommendation. ### Every structural item of this issue is already done — including the one that looked hardest | # | This issue's fix | State | |---|---|---| | 1 | mVoice: whisper load LAZY, not eager-at-startup | **Done.** `server.py` calls `_ensure_ready()` inside the request handlers, and `/api/admin/unload` sets the model back to `None`. Live health on mRock right now: `"loaded":false, "gpu_resident_mib":0`. mVoice no longer eager-loads onto a full GPU, so the OOM crash-loop this issue was written around cannot recur. | | 2 | Route the PWA at the broker | **Done**, `OTTO_PWA_MVOICE_URL` → `mrock:8770/v1/stt`. | | 3 | Big model + name-lexicon biasing | **Done, by a different route.** The broker reaches `large-v3-turbo` on the already-resident whisper-server. Since #10 the lexicon is read from mVoice's own `lexicon.yaml` — the same terms, plus the same alias canonicalization mVoice applies. | | 4 | Verify eviction frees enough for turbo; FLUX reloads | **Moot.** No eviction happens: whisper-server holds ~2050 MiB and coexists with comfyui and ollama. | Acceptance, measured today against production with **m's own reference recordings**, not synthetic audio: ``` DE → "Guten Morgen, mein Name ist Matthias, ich arbeite als Anwalt in Deutschland und baue gerne an verschiedenen Projekten." de, 176 ms EN → "Good morning. My name is Matthias. I work as a lawyer in Germany and enjoy building things with technology." en, 159 ms ``` Verbatim against the reference text, in both languages. ### What #7-full would add if built now A **second copy of the same model**. whisper-server already holds `large-v3-turbo`; item 3 would load `large-v3-turbo` into mVoice as well, on the GPU that is the contended resource. To make that safe it needs the lease-gated load, the eviction round trip and the FLUX-reload verification in items 1 and 4 — engineering whose entire purpose is to survive loading a model the broker can already query in 160 ms without evicting anything. It is also slower by construction. The interim path costs one HTTP hop to a resident model. The #7-full path costs an eviction round trip on the first request whenever FLUX is resident — measured at 12.5 s for a comfyui lease during #5. ### The acoustic slips — they are not #7-full's to fix, and they do not reproduce The slips recorded against full-#7 (`Klappt` heard as `Blatt`, `Satz` as `Nass`) came from espeak-ng clips. Two findings: **They do not reproduce.** Through production today: ``` espeak "Das klappt gut. Ein ganzer Satz zum Testen der Erkennung." → "Das klappt gut. Ein ganzer Satz zum Testen der Erkennung." ``` Verbatim. `klappt` is `klappt`, `Satz` is `Satz`. **And #7-full contains nothing that could have fixed them.** Its scope is the same model (`large-v3-turbo`) with the same lexicon, on a different runtime. Acoustic decoding is not in it. Whatever those slips were — and the espeak artifacts observed since, `mBrian` heard as "im Bayern", suggest the synthesizer rather than the model — they were never evidence for this issue. That distinction was the question, and it decides it: there is no remaining defect that #7-full's scope addresses. ### Two things that are genuinely still open, and are not this issue 1. **mVoice's own STT still runs `small`.** `mvoice.service` launches `--whisper-model small`, and mVoice's `/api/chat`, `/api/respond` and `voice_loop.py` transcribe with it. That is no longer on the PWA path, so it is a small, self-contained model-flag change in `m/mVoice` — and since mVoice already loads lazily, it no longer carries this issue's OOM risk. Worth its own issue **if m still uses those surfaces**; nothing here needs mGPUmanager. 2. **The mDock failover is degraded by design** — whisper `base` on CPU, no lexicon. Different box, no GPU contention, unaffected by anything in #7-full. ### Recommendation **Close as superseded** (m closes issues, not us). Every acceptance bullet of #7 is met on the production path, verified on m's real voice; the one unbuilt item would add VRAM pressure and latency to reach a model the broker already reaches. If m wants mVoice's own STT on a bigger model, that is a one-line change in `m/mVoice` and deserves a fresh issue with its own justification, not this one's OOM framing — which no longer applies, because mVoice already loads lazily.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
No items
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
m
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: m/mGPUmanager#7
No description provided.
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?