Description
Relative trending requires a nuanced model to detect, in closer-to-real-time, whether a page is trending. The data platform engineering team is creating a data product that will serve relative trending.
This ticket focuses on consuming that data and including it in Attribution API responses.
Conditions of acceptance
- Populate the trending.relative signal, using the data from MainStash ( see T432352: [Data Persistence Design Review] Relative trending pages for future reference )
- Determine if a caching strategy is required for storing this data.
- Update schema descriptions to indicate the actual time delay (assumed to be hourly?) through the MVP. Talk with Halley if this is ambiguous or for more information about how specific it should be.
- Notify Halley when this is done
- Update the project page
- Update future audiences
Implementation details
Because this data is being updated on the order of magnitude of hours, we should determine what an appropriate data population and update process might look like. The approach for trending.top ( T418501: [SPIKE?] Attribution API MVP: Determine how to add 'top' trending signal to the attribution trust_and_relevance object ) includes a daily refresh with local cache approach. Determine if something similar should be used here, or if it is preferable to do a direct lookup. If the later, we should still consider implementing a smaller local cache for performance, with the assumption that trending pages are likely to be frequently requested and the data is not frequently changing.
As an additional aside, the current design also assumes that the data will be available in Cassandra and accessible through the Data Gateway, similar to the setup for contributor_counts. In the future, this may evolve to focus on event based updates.