How Medium Powers Real-Time Recommendations at 1M OPS
Inside Medium’s move from relational features to list features in its ScyllaDB-based feature store
“Keep readers reading” is the not-so-simple goal of Medium’s recommendations system. To predict what’s most likely to appeal to a particular reader at any given time, Medium continuously processes user activity signals (stories read, recommendations shown, follows, likes, etc.). It then immediately correlates that with the steady stream of new articles, which is estimated at millions per month.
Smart models and good inference logic are required, but that’s not enough. The data must be stored and retrieved quickly enough to remain relevant while the user is browsing. That’s the job of Medium’s feature store. And getting the data model right started to matter a lot as they scaled to 1M operations per second.
Andréas Saudemont, Medium Principal Software Engineer, recently walked through how the team identified the problem and what they built to fix it....
Copyright of this story solely belongs to hackernoon.com. To see the full text click HERE
