HyVox pairs two kinds of chips to cut the energy cost of speaker identification

A shared audio device needs to recognize enrolled users while treating unfamiliar speakers as unknown. In a November 2026 preprint, Snehasis Addy, Eduardo Calle Ortiz, Prashant Shenoy and Phuc Nguyen of the University of Massachusetts Amherst introduce HyVox, a system that divides this task between a neuromorphic processor and a conventional microcontroller. Their results show an energy–accuracy trade-off, rather than an across-the-board improvement over conventional neural networks.

Two processors, different jobs

HyVox uses a Xylo Audio 3 chip to turn three-second speech windows into compact numerical representations called embeddings. Its spiking neural network processes sound through events and keeps weights close to the neurons using them. The intended benefit is less energy spent moving data between memory and processing hardware.

An STM32L476 microcontroller handles the flexible part: storing enrolled speakers, comparing a query with them and rejecting weak matches. Only 32 spike counts cross from the neuromorphic network to the host for each window. The microcontroller normalizes and centers those counts, then compares the resulting embedding with stored speaker prototypes using cosine similarity, a measure of how closely two vectors align.

Enrollment uses 60 seconds of speech to build a prototype, stored in 64 bytes. Adding or removing someone changes host memory, not the network’s weights. However, updating the embedding network itself would require everyone to enroll again, because their old prototypes would belong to a different representation space.

Making a small network work on real hardware

The backbone contains 832 spiking neurons. Learned synaptic time constants let different neurons integrate evidence over different durations, combining fast acoustic details with slower patterns. A separate 32-neuron readout is trained through episodes that practice enrolling speakers and rejecting outsiders. Backbone training uses VoxCeleb2, while readout training uses a separate speaker population from VoxCeleb1.

Fitting the network onto the chip takes more than reducing its size. The authors compress roughly 188,400 prunable backbone connections to 29,000, then train for integer arithmetic. But hardware also limits events per timestep, delays signals between neurons and clips internal states. Ignoring these details caused a large accuracy drop after mapping the network to the chip.

Their chip-accurate spiking training method, CAST, fine-tunes through a differentiable software replica derived from the chip’s simulator. In a 20-speaker evaluation on previously unseen VoxCeleb1 test speakers, this raised the mapped backbone’s identification accuracy from about 31% to 61%. These are backbone results under a different protocol, not the much higher household identification figures.

Strong identification does not mean reliable rejection

The main household-style evaluation constructs 20 groups from Common Voice recordings, with speakers unseen during both training stages. Members within each group share an accent label, and group sizes are gender balanced. Reported top-1 identification falls from 99.6% with two enrolled speakers to 94.9% with twenty.

Rejection is harder. The rejection equal error rate rises from 5.5% to 13.2% across those sizes. This metric identifies a threshold where the fraction of guests accepted equals the fraction of members rejected. Top-1 separately measures whether a member’s correct prototype ranks highest; it is not a combined measure of correct identification and acceptance.

On twenty additional groups with different speakers, the deployed hardware achieved 96.7% top-1 identification and 10.0% rejection equal error rate with twenty members. These results matched or exceeded the software reference on those groups. The authors suggest hardware-constrained training can improve generalization, but that explanation should not be mistaken for a demonstrated benefit across other recording conditions.

Energy savings, with boundaries

The authors report approximately 0.96 seconds of execution and 1.63 millijoules of dynamic energy per three-second window. Including idle power during execution raises total energy to about 3.8 millijoules. Against BC-ResNet-1, their fastest and most energy-efficient embedded ANN baseline, HyVox used 13.5 times less dynamic energy and 5.8 times less total energy, with 42% shorter execution time. That baseline nevertheless achieved better identification and rejection results.

Execution time is not the complete wait for a decision: HyVox averages three consecutive windows, requiring nine seconds of speech initially and producing an updated decision every three seconds afterward. The measurements include host scoring and idle power but exclude development-kit USB and host-side overhead. They do not establish battery life for a continuously operating product; HyVox also has higher idle power than the tested MCU baselines.

Finally, these are constructed households using recorded speech, not long-term trials in occupied homes. Only about 10% of guest decisions share the household’s accent, and voice changes over years were not studied. The authors position HyVox for personalization, not security-critical authentication without lower errors or a second factor.