Change Details

*Background* Feed is really expensive to build on the fly, so for awhile we tried to store the results in RESTBase until realizing there is just too many problems with it - the daily updates are unpredictable, there's no way to track/observe them. At the times when the feed metadata was stored, we wanted to provide the newest summary content for feed items, so feed was enriched on the fly with summary data. Since it was stored in RESTBase, the code fetching summaries lives in RESTBase too now. *Status quo* After we've gave up on storing feed metadata in RESTBase, the assembly code remained, so now we have a very strange pattern of wikifeeds providing RESTBase urls and RESTBase is calling itself to enrich the response with the summary. This makes no sense. *Solution* We need to port the feed enriching code from RESTBase to WikiFeeds. The code is quite generic, residing mostly in [[ https://github.com/wikimedia/restbase/blob/master/lib/mwUtil.js#L442 | here ]]. The code can't be removed from RESTBase right away since it's used elsewhere (with a less crazy pattern, the urls are assembled in restbase too) Making wikifeeds return full feed response will have the following benefits: - This is the first step in RESTBase sunset project - Makes the request flow much more sane - We can point api-gateway directly to feeds service and stop proxying this one via restbase - RESTBase becomes a simple no-op proxy for feeds just for backwards compatibility. Requests via restbase can be deprecated and traffic slowly migrated to api-gateway. Feed is already exposed there.

**Background** Feed is really expensive to build on the fly, so for awhile we tried to store the results in RESTBase until realizing there is just too many problems with it - the daily updates are unpredictable, there's no way to track/observe them. At the times when the feed metadata was stored, we wanted to provide the newest summary content for feed items, so feed was enriched on the fly with summary data. Since it was stored in RESTBase, the code fetching summaries lives in RESTBase too now. **Status quo** After we've gave up on storing feed metadata in RESTBase, the assembly code remained, so now we have a very strange pattern of wikifeeds providing RESTBase urls and RESTBase is calling itself to enrich the response with the summary. This makes no sense. **Solution** We need to port the feed enriching code from RESTBase to WikiFeeds. The code is quite generic, residing mostly in [[ https://github.com/wikimedia/restbase/blob/master/lib/mwUtil.js#L442 | here ]]. The code can't be removed from RESTBase right away since it's used elsewhere (with a less crazy pattern, the urls are assembled in restbase too) Making wikifeeds return full feed response will have the following benefits: - This is the first step in RESTBase sunset project - Makes the request flow much more sane - We can point api-gateway directly to feeds service and stop proxying this one via restbase - RESTBase becomes a simple no-op proxy for feeds just for backwards compatibility. Requests via restbase can be deprecated and traffic slowly migrated to api-gateway. Feed is already exposed there.

***Background*** Feed is really expensive to build on the fly, so for awhile we tried to store the results in RESTBase until realizing there is just too many problems with it - the daily updates are unpredictable, there's no way to track/observe them. At the times when the feed metadata was stored, we wanted to provide the newest summary content for feed items, so feed was enriched on the fly with summary data. Since it was stored in RESTBase, the code fetching summaries lives in RESTBase too now. ***Status quo*** After we've gave up on storing feed metadata in RESTBase, the assembly code remained, so now we have a very strange pattern of wikifeeds providing RESTBase urls and RESTBase is calling itself to enrich the response with the summary. This makes no sense. ***Solution*** We need to port the feed enriching code from RESTBase to WikiFeeds. The code is quite generic, residing mostly in [[ https://github.com/wikimedia/restbase/blob/master/lib/mwUtil.js#L442 | here ]]. The code can't be removed from RESTBase right away since it's used elsewhere (with a less crazy pattern, the urls are assembled in restbase too) Making wikifeeds return full feed response will have the following benefits: - This is the first step in RESTBase sunset project - Makes the request flow much more sane - We can point api-gateway directly to feeds service and stop proxying this one via restbase - RESTBase becomes a simple no-op proxy for feeds just for backwards compatibility. Requests via restbase can be deprecated and traffic slowly migrated to api-gateway. Feed is already exposed there.