I Turned My Eyes Into a Game Controller, With Nothing But a Webcam

Gaze control has always felt like a luxury feature. The good hardware, such as Tobii bars, dedicated IR trackers, costs more than the games you'd play with it, and every tutorial seems to assume you already own one. So I asked the obvious question: how far can I get with the webcam that's already sitting on top of my monitor?

The answer, it turns out, is surprisingly far. Far enough to look around a 3D scene by moving my eyes. Far enough to aim a shooter with my gaze. And the whole thing runs on four Python libraries you can pip install in about ninety seconds.

Here's how I built it, what actually made it work, and, just as importantly, where the illusion breaks down.

The core idea: iris landmarks, not eye-tracking hardware

The trick that makes this possible is MediaPipe's Face Meshwith refine_landmarks=True. That flag unlocks a...

Copyright of this story solely belongs to hackernoon.com. To see the full text click HERE