Page MenuHomePhabricator

Design Document that proposes an alternative architecture for historic data endpoints
Closed, ResolvedPublic8 Estimated Story Points

Event Timeline

fdans moved this task from Incoming to Smart Tools for Better Data on the Analytics board.

@WDoranWMF should this task be in the To Do column as there's no one assigned to it?

My apologies cause i though this task had been updated a while back, here is the document: https://docs.google.com/document/d/1IacO1l5Y0uGrVnOZQ-3JskZaRqtiwqvpAffmAlJZquQ/edit

This task is not fully finished, we have halted progress due to resourcing on covifd issues, @Milimetric to provide an update

We basically split this effort into two phases. The design for the first, designing a better data pipeline which would feed highly available historic data endpoints, is done. The second phase is a bit too abstract so I'm going to break it out into a separate task. For now, I'll take this, polish up anything that needs polishing on the document, and call it done. We can then implement whenever Core Platform is ready to work together on it.

Thank you this is important work~ Kudos to the team.
Can we make https://docs.google.com/document/d/1IacO1l5Y0uGrVnOZQ-3JskZaRqtiwqvpAffmAlJZquQ/edit accessible to general reader? I am interested in reading it.

I don't see why not, making it read-only now. The first section on the lambda architecture is considered reviewed and done, we're still playing around with the second section.

read-only link: https://docs.google.com/document/d/1IacO1l5Y0uGrVnOZQ-3JskZaRqtiwqvpAffmAlJZquQ/edit?usp=sharing

Nuria set the point value for this task to 8.