Page MenuHomePhabricator
Create Task
Maniphest T263780

Move feeds from behind RESTBase to the API Gateway
Closed, ResolvedPublic1 Estimated Story Points
Actions

Description

As a part of T262315 we need to transfer the wikifeeds service from behind RESTBase. T263133 has dramatically simplified the RESTBase portion of the feed assembly, so there's not much left for RESTBase to do.

Here's the plan:

  1. Review the wikifeeds service API compared to feed exposed by RESTBase. The wikifeeds service is currently lacking the aggregate endpoint (the /feed/featured in RESTBase), it should be added. In general, we will be capable of remapping parameters in the gateway slightly, but it's better to make the service expose the API as close to the gateway API. The whole service will be exposed under a single prefix, e.g. `/feed/.
  2. Implement any modifications to the wikifeeds API, together with modifying RESTBase backend requests
  3. Expose wikifeeds over API Gateway, deprecate RESTBase feeds API
  4. Eventually, drop RESTBase feeds API.

Related Objects
Search...

Event Timeline

• Pchelolo created this task.Sep 24 2020, 6:16 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 24 2020, 6:16 PM
• Pchelolo edited parent tasks, added: T262315: <CORE TECHNOLOGY> API Migration & RESTbase Sunset; removed: T263133: Move feed assembly from RESTBase to Wikifeeds.Sep 24 2020, 6:20 PM
• Pchelolo added a project: Platform Engineering Roadmap.Sep 24 2020, 9:10 PM
WDoranWMF added a project: Code-Health.Sep 25 2020, 1:45 PM
• Mholloway subscribed.Sep 25 2020, 2:07 PM
WDoranWMF edited projects, added Code-Health-Objective; removed Code-Health.Sep 25 2020, 7:54 PM
daniel moved this task from Inbox to Future Initiatives/Small Projects on the Platform Engineering board.Sep 29 2020, 8:28 PM
LGoto moved this task from Needs triage to Tracking on the Product-Infrastructure-Team-Backlog-Deprecated board.Sep 30 2020, 3:45 PM
• Mholloway added a comment.Sep 30 2020, 3:46 PM

Just confirming that Platform Engineering will be handling this, is that correct?

• Pchelolo added a comment.Sep 30 2020, 8:46 PM
In T263780#6505626, @Mholloway wrote:

Just confirming that Platform Engineering will be handling this, is that correct?

Might need some code review or get your opinions on something from time to time, but generally yes.

Naike added subscribers: WDoranWMF, Naike.Oct 2 2020, 9:36 AM

@WDoranWMF, would it make sense to create a vision table for this initiative, so we're documenting our work consistently on MediaWiki?

Naike set the point value for this task to 1.Oct 2 2020, 9:42 AM
WDoranWMF moved this task from Later (future inbox) to Next on the Platform Engineering Roadmap board.Oct 5 2020, 4:21 PM
WDoranWMF moved this task from Next to Later (future inbox) on the Platform Engineering Roadmap board.
WDoranWMF moved this task from Later (future inbox) to Next on the Platform Engineering Roadmap board.Oct 5 2020, 4:58 PM
WDoranWMF moved this task from Next to Later (future inbox) on the Platform Engineering Roadmap board.Oct 5 2020, 5:09 PM
WDoranWMF moved this task from Later (future inbox) to Next on the Platform Engineering Roadmap board.Oct 5 2020, 8:24 PM
WDoranWMF moved this task from Next to Later (future inbox) on the Platform Engineering Roadmap board.
Naike moved this task from Later (future inbox) to Next on the Platform Engineering Roadmap board.Oct 6 2020, 5:36 PM
WDoranWMF removed projects: Platform Team Initiatives (API Gateway), Platform Engineering.Oct 9 2020, 4:19 PM
• sdkim moved this task from Backlog to Verify on the Wikifeeds board.Oct 21 2020, 5:38 PM
• sdkim added a parent task: T266457: [EPIC] Migrate Wikifeeds from RESTbase to API Gateway.Oct 28 2020, 2:03 PM
Peter.ovchyn claimed this task.Nov 2 2020, 5:42 PM
Peter.ovchyn removed Peter.ovchyn as the assignee of this task.Nov 17 2020, 1:21 PM
Peter.ovchyn subscribed.
Naike moved this task from Next to Later (future inbox) on the Platform Engineering Roadmap board.Dec 7 2020, 6:39 PM
BPirkle mentioned this in T303822: Evaluate API Platform Tasks Related to RestBase Sunset.Mar 17 2022, 6:13 PM
• ACCDCCDB subscribed.Oct 13 2022, 2:41 PM
• ACCDCCDB merged a task: T320730: Define access to external resources for GitLab CI Runners.Oct 13 2022, 2:43 PM
• ACCDCCDB added subscribers: Jelto, thcipriani, LSobanski and 7 others.
Aklapper removed subscribers: akosiaris, Joe, Dzahn and 11 others.Oct 13 2022, 5:03 PM
Jgiannelos subscribed.Dec 5 2022, 6:40 PM

After checking the codebase on how wikifeeds is defined on restbase these are the findings:

  • wikifeeds endpoint are defined in feed.js
  • it is built in a way to combine multiple requests to wikifeeds endpoints (/page/featured, /page/most-read, /media/image/featured, /page/news, /feed/onthisday) to a single response
  • On the backend service side the outgoing requests to restbase are
    • /page/html (parsoid)
    • /page/summary (mobileapps)
    • aqs metrics

In order to migrate the feeds endpoint outside of RESTBase we need to point the backend service to the alternative services:

  • Parsoid on MW REST Api
  • Sumarry on mobileapps
  • TBD for AQS

On top of that, the custom response defined on feed.js needs to be re-implemented in the wikifeeds service level.

Jgiannelos added a subscriber: daniel.Dec 5 2022, 6:40 PM
MSantos edited projects, added RESTBase Sunsetting, Content-Transform-Team-WIP; removed Product-Infrastructure-Team-Backlog-Deprecated.May 18 2023, 2:15 PM
MSantos moved this task from Backlog to Current Epics on the Content-Transform-Team-WIP board.
daniel triaged this task as Medium priority.Jun 5 2023, 6:14 PM
Jgiannelos closed subtask T339117: Benchmark performance of wikifeeds service responses compared to restbase in production as Resolved.Jul 11 2023, 8:16 AM
Jgiannelos closed subtask T339118: Run difftesting between wikifeeds service and restbase endpoints as Resolved.Jul 13 2023, 2:10 PM
Jgiannelos closed subtask T336758: Move feeds logic implemented on restbase to wikifeeds service as Resolved.Jul 17 2023, 3:21 PM
MSantos moved this task from Unsorted to Doing on the RESTBase Sunsetting board.Aug 18 2023, 3:21 PM
MSantos assigned this task to Jgiannelos.Sep 18 2023, 9:23 AM
MSantos moved this task from Current Epics to To Verify on the Content-Transform-Team-WIP board.
Jgiannelos closed this task as Resolved.Sep 21 2023, 11:27 AM
Jgiannelos closed subtask T339119: Switchover plan from restbase to api gateway for wikifeeds as Resolved.
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL