Warren
Say the phrase and audio streams to a server in the house, which transcribes it, asks a model and answers out loud. Nothing leaves the building. The same microphone feeds a listener that groups speech into conversations and writes each one up.
- Board
- ESP32-S3 and a MEMS microphone on a 58 by 42 mm board of my own
- Wake phrase
- Trained from scratch on synthesized speech. 62 KB.
- Pipeline
- Voice detection, Whisper, speaker embedding, grouped on a three-minute gap
- Summaries
- Run on the local CLI tool, not a metered API
Timestamps come off the queue
Audio is stamped when it goes onto the transcription queue, not when transcription finishes, so a conversation is grouped by when people spoke rather than by when the transcriber caught up with them.
Speaker identification
Named voices are matched at a higher similarity threshold than anonymous clusters, and have to beat the best anonymous cluster by a margin before they are claimed.
Labelling a voice and learning from it are two separate decisions, so a match that is good enough to show is not automatically good enough to train on.
Diarization was rejected on the numbers
Proper diarization ran at nine tenths of real time on this hardware, against six hundredths for speaker embedding alone, and it invented dozens of speakers.
The board is generated, not drawn
Schematic, layout, routing, bill of materials and assembly files all come out of Python. The generator is edited; the layout never is. The back carries a line drawing exposed as bare copper through the solder mask.
