Page MenuHomePhabricator
Create Task
Maniphest T314025

[EPIC] Migrate PCS service away from restbase
Closed, ResolvedPublic
Actions

Description

PCS, the service that is used for mobile apps to consume wiki content (+ some other functionality), is deployed using restbase. We would like to move this app away from having restbase a dependency.

This is a tracker ticket for this effort.

Acceptance criteria
  • PCS is serving production traffic with no dependency to restbase

Related Objects
Search...

StatusSubtypeAssignedTask
 StalledNoneT324931 Clean up open RESTBase related tickets
 OpenNoneT262315 <CORE TECHNOLOGY> API Migration & RESTBase Sunset
 ResolvedDbrantT328943 Replace PCS lazy-loading logic with standard "loading=lazy" attribute
 ResolvedMSantosT314025 [EPIC] Migrate PCS service away from restbase
 ResolvedJgiannelosT314764 Branch out mobileapps for restbase deprecation
 ResolvedJgiannelosT314765 Replace RESTBase endpoint references of outgoing API calls
 ResolvedJgiannelosT314766 Use core page HTML as a replacement for RESTBase parsoid calls
ResolvedJgiannelosT314767 Log outgoing RESTBase API calls
 ResolvedJgiannelosT314768 [spike] Performance benchmarking of mobileapps without RESTBase
 ResolvedJgiannelosT314770 [spike] Performance benchmarking of RESTBase/mobileapps without cassandra
 ResolvedJgiannelosT323514 Disable storage for testwiki PCS on restbase production
 ResolvedJgiannelosT329837 Find a wiki to disable cassandra storage for `mobile-html` endpoint
 ResolvedJgiannelosT314781 Perfomance baseline metrics for mobileapps
 ResolvedJgiannelosT314782 Create a dataset of wiki pages to run as a test/benchmark suite
 ResolvedMSantosT314771 Implement a compatibility layer between RESTBase and native PCS responses
 ResolvedMSantosT321194 Implement Security Response Header Filter logic outside RESTBase for PCS
 ResolvedJgiannelosT321195 Implement Access Check Filter outside RESTBase for PCS
 OpenNoneT321199 Implement Ensure Content Type Filter logic outside RESTBase for PCS
 ResolvedJgiannelosT314777 Investigate cache purging needs
 ResolvedJgiannelosT321101 Investigate moving summary endpoint outside of mobileapps service
 ResolvedJgiannelosT264670 Move PCS endpoints behind API Gateway
 ResolvedhnowlanT372746 hewiki: Route mobile-html to the backing node service instead of RESTBase
 ResolvedJgiannelosT372749 hewiki: Use backing node service instead of RESTBase on pregeneration changeprop rules
 ResolvedJgiannelosT385719 testwiki: Add changeprop rule to send requests to PCS on resource change
 ResolvedJgiannelosT385821 Dont propagate server error details to end users
 ResolvedJgiannelosT387277 Rollout more wikis after week 1 of testing with production traffic
 ResolvedJgiannelosT388214 Pregeneration rules don't pregenerate caches for the same cases restbase did
 ResolvedJgiannelosT388140 Rollout more wikis: week 2
 ResolvedJgiannelosT385033 Switchover plan from restbase to rest gateway for PCS endpoints that don't require cache
 ResolvedWRai-WMFT389073 PCS switchover: test editing on testwiki and/or User namespace
 ResolvedJgiannelosT390080 Accept language is not working for srwiki
 ResolvedJgiannelosT390724 Rollout more wikis: week 3
 ResolvedTsevenerT389074 [Testing] PCS switchover: test editing on testwiki and/or User namespace
 ResolvedJgiannelosT392052 PCS caching metrics: Add label to distinct production vs pregeneration traffic
 OpenBUG REPORTNoneT363279 /page/summary/ and /page/mobile-html/ should handle BCP47 languge code for wikidata description
 ResolvedMSantosT392491 Page talk endpoint is not used by any of the apps
 ResolvedJgiannelosT393591 Rollout more wikis: week 4
 OpenNoneT321207 Core REST endpoints should cover needs of node services
 ResolvedMSantosT301372 Core Page REST API should follow redirects
 ResolvedTK-999T290639 Move redirect lookup logic into a service object
 DuplicateNoneT321190 page/{title}/HTML endpoint should support "virtual" content on message pages
 Resolvedabi_T317019 REST page/html endpoints should support language variants
 DuplicateNoneT262593 MediaWiki Developer stores different HTML output for different language variants in the parser cache
 Resolvedabi_T318401 Language conversion in page/html endpoints need to fall back to old LanguageConverter classes of Parsoid doesn't support the conversion.
 OpenNoneT322426 Create REST endpoint for getting the latest "stable" HTML
OpenNoneT326742 REST API: Wiki redirects include an HTML representation of the redirect in the body of the response
 OpenNoneT326743 REST API: include a machine readable representation of redirect targets in responses
 InvalidNoneT322152 Route external requests to RESTbase via API gateway
 ResolvedJgiannelosT333343 Cache control header configuration on restbase is not working for PCS endpoints
 OpenNoneT338126 Spike: Investigate LanguageConverter redirects for language variants
 InvalidJgiannelosT340129 PCS performance optimization to avoid the need for pregeneration
 ResolvedJgiannelosT343120 Investigate cache needs based on existing logs/metrics
 ResolvedJgiannelosT344943 Determine how PCS will be informed about resource changes in a world without RESTbase
 ResolvedJgiannelosT319365 PCS caching and pregeneration when restbase is decommissioned
 ResolvedcoolteyT322142 Measure network latency of page load requests (Android)
 ResolvedSNowick_WMFT327548 Report network latency of page load requests (Android)
 ResolvedSNowick_WMFT332769 pt.wiki latency P95 + P99 in apps pre- and post-disabling of pregeneration in PCS
 ResolvedEevansT348993 Create new cassandra table data model for PCS
 ResolvedJgiannelosT348995 Introduce PCS cache management layer
ResolvedhnowlanT350507 Update mobileapps k8s deployment chart for Cassandra credentials
 ResolvedJgiannelosT348996 Change changeprops rules to pre-generate/invalidate cache directly to PCS rather than in restbase
 ResolvedJgiannelosT366819 Enable PCS to send resource change events to handle URL purges
 ResolvedJgiannelosT368052 Allow connections from PCS to eventgate
 ResolvedJgiannelosT385706 Move cache control headers emitted by restbase to PCS
 ResolvedJgiannelosT385718 PCS raises errors when trying to send requests to eventgate

Event Timeline

Jgiannelos created this task.Jul 28 2022, 8:43 AM
Restricted Application added a project: Product-Infrastructure-Team-Backlog-Deprecated. · View Herald TranscriptJul 28 2022, 8:43 AM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
Jgiannelos triaged this task as High priority.Jul 28 2022, 8:44 AM
Jgiannelos updated the task description. (Show Details)
Jgiannelos updated the task description. (Show Details)
Aklapper added a comment.Aug 8 2022, 10:34 AM

(Please semnatically connect dependencies - adding T262315: <CORE TECHNOLOGY> API Migration & RESTBase Sunset as a parent task.)

Aklapper added a parent task: T262315: <CORE TECHNOLOGY> API Migration & RESTBase Sunset.Aug 8 2022, 10:34 AM
• DAlangi_WMF subscribed.Aug 16 2022, 3:08 PM
Jgiannelos moved this task from Needs triage to PCS/RESTBase sunset on the Product-Infrastructure-Team-Backlog-Deprecated board.Sep 22 2022, 3:11 PM
Jgiannelos closed subtask T314768: [spike] Performance benchmarking of mobileapps without RESTBase as Resolved.Oct 5 2022, 10:11 AM
daniel added a subtask: T264670: Move PCS endpoints behind API Gateway.Oct 19 2022, 2:53 PM
daniel added a project: Platform Team Workboards (MW Expedition).Oct 19 2022, 3:04 PM
daniel added a project: Epic.
daniel moved this task from Unsorted pile to Doing on the Platform Team Workboards (MW Expedition) board.
daniel added a subtask: T322152: Route external requests to RESTbase via API gateway.Nov 1 2022, 4:00 PM
Jgiannelos added a subtask: T322142: Measure network latency of page load requests (Android).Nov 1 2022, 4:28 PM
Jgiannelos changed the status of subtask T321101: Investigate moving summary endpoint outside of mobileapps service from Open to In Progress.Nov 3 2022, 11:40 AM
Jgiannelos closed subtask T314764: Branch out mobileapps for restbase deprecation as Resolved.Nov 4 2022, 6:37 PM
Jgiannelos closed subtask T321101: Investigate moving summary endpoint outside of mobileapps service as Resolved.Nov 22 2022, 4:14 PM
daniel added a project: RESTBase Sunsetting.Dec 6 2022, 3:35 PM
daniel moved this task from Unsorted to Doing on the RESTBase Sunsetting board.Dec 6 2022, 3:39 PM
Dbrant closed subtask T322142: Measure network latency of page load requests (Android) as Resolved.Dec 8 2022, 4:53 PM
MSantos mentioned this in T226931: Parsoid cache invalidation for mobile-sections seems not reliable.Jan 9 2023, 4:42 PM
Jgiannelos closed subtask T314765: Replace RESTBase endpoint references of outgoing API calls as Resolved.Jan 18 2023, 11:12 AM
Jgiannelos closed subtask T314767: Log outgoing RESTBase API calls as Resolved.Feb 16 2023, 5:14 PM
hnowlan changed the status of subtask T322152: Route external requests to RESTbase via API gateway from Open to In Progress.Mar 23 2023, 3:24 PM
Jgiannelos closed subtask T333343: Cache control header configuration on restbase is not working for PCS endpoints as Resolved.Mar 29 2023, 1:58 PM
MSantos removed a project: Product-Infrastructure-Team-Backlog-Deprecated.May 8 2023, 7:43 AM
MSantos moved this task from Needs Triage to In Progress on the Page Content Service board.
daniel moved this task from Doing to Blocked on the RESTBase Sunsetting board.Jun 5 2023, 6:05 PM
Jgiannelos added a subtask: T338126: Spike: Investigate LanguageConverter redirects for language variants.Jun 8 2023, 2:30 PM
MSantos renamed this task from Migrate PCS service away from restbase to [EPIC] Migrate PCS service away from restbase.Jun 12 2023, 11:11 AM
MSantos added a project: Content-Transform-Team-WIP.
MSantos moved this task from Backlog to Current Epics on the Content-Transform-Team-WIP board.
MSantos added a subtask: T348993: Create new cassandra table data model for PCS.Oct 16 2023, 2:33 PM
MSantos added a subtask: T319365: PCS caching and pregeneration when restbase is decommissioned.Oct 16 2023, 2:49 PM
MSantos removed a subtask: T348993: Create new cassandra table data model for PCS.
MSantos moved this task from Blocked to PCS Service Pile on the RESTBase Sunsetting board.Oct 24 2023, 11:04 AM
Jgiannelos closed subtask T314777: Investigate cache purging needs as Resolved.Jan 5 2024, 1:05 PM
TheDJ added a parent task: T328943: Replace PCS lazy-loading logic with standard "loading=lazy" attribute.Jan 24 2024, 8:40 AM
Jgiannelos closed subtask T344943: Determine how PCS will be informed about resource changes in a world without RESTbase as Resolved.Oct 28 2024, 3:20 PM
MSantos closed subtask T314771: Implement a compatibility layer between RESTBase and native PCS responses as Resolved.Nov 21 2024, 11:27 AM
Jgiannelos closed subtask T385706: Move cache control headers emitted by restbase to PCS as Resolved.Feb 6 2025, 1:57 PM
Jgiannelos closed subtask T319365: PCS caching and pregeneration when restbase is decommissioned as Resolved.Feb 7 2025, 3:30 PM
Jgiannelos closed subtask T385718: PCS raises errors when trying to send requests to eventgate as Resolved.Feb 20 2025, 12:03 PM
MSantos closed subtask T322152: Route external requests to RESTbase via API gateway as Invalid.Mar 12 2025, 1:02 PM
Jgiannelos edited projects, added: Content-Transform-Team (Work In Progress); removed: Content-Transform-Team-WIP, Platform Team Workboards (MW Expedition).Apr 11 2025, 1:11 PM
Jgiannelos moved this task from Backlog to Current Epics on the Content-Transform-Team (Work In Progress) board.
Jgiannelos closed subtask T264670: Move PCS endpoints behind API Gateway as Resolved.Jul 7 2025, 3:10 PM
MSantos removed a subtask: T386919: Pregeneration performance optimizations for PCS.Jul 30 2025, 8:07 AM
MSantos closed this task as Resolved.Sep 4 2025, 2:29 PM
MSantos claimed this task.
Jgiannelos closed subtask T340129: PCS performance optimization to avoid the need for pregeneration as Invalid.Wed, May 13, 8:40 AM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits