Continuous Aggregate Refresh, Demystified: Invalidation, Lookback, and Late-Arriving Data
Yesterday the shift dashboard reported 41,900 units across an eight-hour window. Today the same query over the same eight hours reports 43,100. Nobody shipped a code change and nobody edited a row after the fact, so why did the number move? A plant-floor historian had lost its uplink, buffered locally, and flushed overnight, landing rows that carry their original timestamps. The raw table was correct at every point along the way. The pre-computed aggregate behind the dashboard never went back for those rows, so it published one number and later replaced it with another.
Every pre-computation strategy handles a fresh row arriving at the head of the table. They diverge on the two events no evaluation demo covers: data that arrives late for a window already computed, and data that changes after the fact. This piece runs the four common strategies against exactly those events to show why each behaves...
Copyright of this story solely belongs to hackernoon.com. To see the full text click HERE
