FRIDAI, voice assistant for gamers

The context

A consumer gaming product, built to ship publicly on players' own PCs rather than run as an internal tool. The operating environment was hostile to voice software: game audio and teammate chat in the same room, a CPU and GPU already saturated by the game, and users who abandon anything that costs frames or focus. The assistant had to stay always listening, respond at play speed, and hold attention after the novelty wore off. The same constraints now define enterprise voice agents.

Why normal software was not enough

Overlays and hotkeys already existed; the problem was that a player's hands and eyes are fully occupied. The interface had to be free speech, and free speech is not something rules can parse: gaming vocabulary, boss names, and slang defeat fixed command grammars. Wake-word detection, streaming speech recognition, and intent understanding are learned capabilities. And answering open game questions mid-match needs language understanding, not a lookup table.

The problem

Gamers cannot alt-tab mid-match. Anything that helps them, from clipping a highlight to looking up a boss fight, has to work by voice, in seconds, without stealing focus from the game.

What we built

We built FRIDAI as a PC-native voice companion: wake-word detection, streaming speech recognition tuned for gaming vocabulary, and an action layer that executed game-related tasks like capturing clips, launching music, and answering game questions, all without interrupting the session.

What production made hard

• Latency was the product. The full chain, wake word to speech recognition to intent to action, had to finish fast enough to use mid-match, which meant streaming every stage instead of waiting for complete utterances.
• The audio environment worked against us: game sound, teammates on voice chat, and mechanical keyboards, with a wake word that had to fire reliably without triggering on the chaos.
• The assistant shared a PC with a game that wanted every CPU and GPU cycle. Anything that cost frames would be uninstalled, so the always-on components had to stay lightweight.
• Generic speech models mishear gaming vocabulary. Game titles, boss names, and player slang forced recognition tuned to the domain rather than taken off the shelf.
• Mid-match, there is no time for a clarification dialogue. Misheard commands had to fail cheap: act only when confident, decline when not, and keep every action easy to reverse.

What a similar project needs

• Recorded audio from the real environment, not a quiet room: the accents, background noise, and domain vocabulary your users actually produce, plus a test set of phrasings to measure recognition against.
• A timeline that front-loads proof: a working voice-to-action loop in about scoped, our Prototype Sprint shape, then most of the schedule spent tuning latency and vocabulary with real users.
• One team across the seam: speech and model engineering sitting with the client and product engineers, because latency and reliability die in handoffs between vendors.
• A latency budget treated like a test: measured per release, end to end, so regressions fail the build instead of reaching users.
• An honest retention risk: voice interfaces survive only where hands and eyes are busy and the assistant beats the alternative. Prove people keep using it after the novelty wears off before scaling.