On-device transcription for product

Goal

Some products sit in an awkward spot: cloud AI costs more per use than the feature earns, yet users genuinely want the feature. For those products, running the AI on the device itself is the only math that works.

Report

The opening came from a real product team with exactly that problem for transcription. Their standalone demo got a new engine in an afternoon: a small speech model running fully on-device, instructed into place by a coding agent. It works, including on the Android tablets used for product testing.

Working prototype: yes. Product decision: still pending.