On-Device Body Tracking in a PWA: How AURADUEL Scores an Aura Battle Without a Server
In case you missed it:
Our players are 16 to 24 years old. Many are minors. So the first rule of the project was written before the first line of code: biometrics and movement analysis run 100% on the device, and no body landmark ever leaves the phone. There is no server that sees a body. There is nothing to leak.
That rule forced every other decision in this article.
The stack
- React 18 + Vite + TypeScript, shipped as a PWA and wrapped with Capacitor 8 for iOS.
- @mediapipe/tasks-vision 1.0.1: GestureRecognizer for the hands, PoseLandmarker lite (float16) for the body.
- pixi.js 8 for the AR layer: spheres, sparks, and the verdict drawn on top of the camera in real time.
- Firebase: Hosting, Firestore, and Realtime Database for the mirror duel between two phones.
- RevenueCat (purchases-capacitor) for in-app purchases.
Loading 15 MB of vision models for free
The WASM engine...
Copyright of this story solely belongs to hackernoon.com. To see the full text click HERE
