I Added Every Index, but My Telemetry Pipeline Was Still Quadratic

https://hackernoon.imgix.net/images/5wpKgV75aONqkTJlafw2yQmK9yd2-ax83b9l.png

A 15-minute FPV flight is 9,000 GPS points. My ingest path ran 40 million distance calculations to keep one running total up to date.

Last time, I wrote about SearchWork, where normalizing eleven job-board APIs turned out to be harder than any of the LLM work. This one is about a different project and a different lesson: the difference between a slow query and a slow algorithm, and why adding indexes fixes only one of them.

The project is FLYON - a web platform for drone and FPV owners. It takes live telemetry from a ground station, draws the flight on a map, warns you when you enter a no-fly zone, and scores the flight afterwards. Node/Express, PostgreSQL with PostGIS, WebSockets for the live view, Next.js on the front.

The part that broke is the part I thought was trivial.

9,000 points

The ground bridge is a small Python...

Copyright of this story solely belongs to hackernoon.com. To see the full text click HERE

Read more