Case Study · Wearable HCI
LYRA
A wrist-worn gesture interface that turns hand movements into media control — with all of the classification running on the microcontroller itself.
Built · Working
The problem
Controlling music while your hands are busy — cooking, carrying things, walking — means fumbling for a phone or a laptop. I wanted a wearable that would let a flick of the wrist skip a track, without a companion app doing the thinking, and without firing accidentally every time I moved my arm.
The second half of that sentence is the actual engineering problem. A gesture recognizer that triggers falsely destroys trust in about a day; and one that only works for the person who trained it isn't a product, it's a demo.
Constraints
- On-device inference. Classification runs on the ESP32-C3 itself — no phone or laptop in the loop for recognition. That rules out heavy models and forces a small, cheap feature set.
- False positives matter more than accuracy. An unprompted "next track" mid-gesture-less motion is worse than a missed gesture, so the pipeline had to reject everyday arm movement aggressively.
- Cross-user generalisation. I evaluated with leave-one-subject-out splits — the model is always tested on a person it has never seen.
- Wrist-sized hardware. ESP32-C3 + MPU-6050, housed in a custom 3D-printed two-story wrist enclosure.
System architecture
The MPU-6050 streams 6-axis accelerometer + gyroscope data to the ESP32-C3, which runs a two-stage classifier:
- Stage 1 — rule gate. Cheap rule-based checks reject the vast majority of non-gesture motion before any model runs.
- Stage 2 — decision tree. Candidate windows are summarised into 9 IMU features and classified by a decision tree — small enough to run comfortably on the microcontroller.
Recognised gestures are emitted as BLE-HID media commands, so LYRA pairs with a phone or laptop like a Bluetooth keyboard — no driver, no app required on the host. A Flutter app handles per-user calibration, and a bridge mode retargets gestures to drive a LEGO EV3 robot arm instead of media keys.
MCU
ESP32-C3 (RISC-V)
IMU
MPU-6050 · 6-axis accel + gyro
Classifier
Rule gate + decision tree · 9 IMU features · on-device
Output
BLE-HID media control · EV3 robot-arm bridge
Calibration
Flutter companion app
Enclosure
Custom 3D-printed two-story wrist case
What I built
- ESP32-C3 firmware: IMU sampling, the rule gate, feature extraction, decision-tree inference, and the BLE-HID stack.
- The dataset: 419 labelled samples from 4 subjects, collected under a consistent protocol.
- A reproducible analysis pipeline — feature engineering, training and leave-one-subject-out evaluation can be re-run end to end.
- The Flutter calibration app.
- The LEGO EV3 robot-arm bridge, and the 3D-printed wrist enclosure.
Results
| Metric | Value |
|---|---|
| Cross-user accuracy (leave-one-subject-out), overall | 79% |
| Cross-user accuracy, flick gesture | 99% |
| False-positive rate | 10% |
| Dataset | 419 samples · 4 subjects |
| Evaluation | Leave-one-subject-out · reproducible pipeline |
The headline for me is the evaluation method, not the number: 79% is what a new user gets before any calibration, because every test subject was held out of training. The flick — the primary "skip track" gesture — is essentially solved at 99%.
Honest limitations
- Four subjects is a small dataset. 419 samples across 4 people supports the cross-user claim directionally, not conclusively; more subjects would almost certainly move the numbers.
- 79% overall cross-user accuracy is not product-grade. That's why the Flutter calibration app exists — per-user calibration is still part of the story.
- A 10% false-positive rate is honest, but too high for all-day wear. The rule gate helps; it doesn't fully solve it.
- Results come from a controlled collection protocol, not from days of unconstrained daily use.