Page MenuHomePhabricator
Create Task
Maniphest T177425

Develop General Layer of PCS
Closed, ResolvedPublic
Actions

Description

This task is for tracking the work in the General (middle) layer of the PCS.

This layer is focused on:

  1. Providing structured APIs for general client consumption of page data
  2. Consolidating data and logic from different APIs needed by clients into one consistent set of APIs (RESTBase + Wikipedia + Commons + Wikidata)
  3. Applying common JS transforms to HTML that most clients typically perform independently
  4. Providing a set of components for the client specific layer to use for creating client specific APIs, minimizing the processing needed to be done in that layer

Please see the subtask for the specific API implementations that make up this layer and see the parent ticket for details at the system level.

Related Objects
Search...

StatusSubtypeAssignedTask
 ResolvedNoneT169242 Develop Page Content Service for Reading Clients
 ResolvedNoneT177425 Develop General Layer of PCS
 Resolved• JhernandezT177426 Develop structured JSON APIs for general consumption
 Resolved• bearNDT170690 Extract a References JSON API
 Resolved• bearNDT146398 Create Mobile content service endpoint for references
 DeclinedNoneT146396 Improve performance of mobile references API
 Resolved• FjalapenoT170294 Question: Do clients prefer to have references as structured data in order to build more useful UIs (rather than just showing a bunch of references at the bottom of the page)
 Resolved• bearNDT174810 Add swagger spec for references endpoint
 Resolved• bearNDT182647 Backlinks should be objects with href and the link text
 Resolved• bearNDT182652 Citations field becomes type with just a single value
 Resolved• bearNDT184327 Add section header to references output
 Resolved• bearNDT206467 Provide all reference lists
 Resolved• bearNDT184328 Add revision + tid to references output
 Resolved• bearNDT185133 Update references wrapper lookup
 Resolved• bearNDT185138 Update unit tests to account for Parsoid ref change from span to sup
 Resolved• bearNDT190219 Make section_heading part of the reference_list
 Resolved• MhollowayT177428 Develop Metadata JSON API
 Resolved• MhollowayT172005 Extract categories as a structured array in the metadata endpoint
 Resolved• MhollowayT172004 Extract Hatnotes as a structured array
 Resolved• MhollowayT186838 Handle dewiki hatnotes
 Resolved• MhollowayT186834 Selfref template produces nested div.hatnote elements, causing duplicate hatnote output in MCS
 Resolved• MhollowayT172002 Extract Page Issues as a structured array
 Resolved• MhollowayT179947 Improve geo coordinate reporting in MCS
 Declined• MhollowayT187841 Decide on protection output format for the metadata endpoint
 Resolved• MhollowayT188563 Document the metadata endpoint in an on-wiki spec
 Resolved• MhollowayT177430 Develop a Media JSON API
 Resolved• MhollowayT179876 Expose media endpoint in RESTBase when ready
 DuplicateNoneT179878 Security review of the MCS media endpoint
 Resolved• MhollowayT86954 Service: Image gallery must correspond one-to-one with images displayed on page (in order and visibility).
 DeclinedNoneT182329 Media: handle items that link to documents other than a File page
 Resolved• MhollowayT182330 Media: handle galleries
 Resolved• MhollowayT188562 Document the media endpoint in an on-wiki spec
 Resolved• MhollowayT177431 Develop a Summary JSON API
 ResolvedovasilevaT113094 [EPIC] The Page Summary API needs to provide useful content for the majority of articles
 ResolvedphuedxT113633 Spike: Alternative to TextExtracts for Popups, Gather, Read more
 ResolvedphuedxT109867 Infobox template parameters showing in Hovercard
 DuplicateNoneT109869 Hovercards should not show <noinclude> content in the extract
 ResolvedJdlrobsonT74629 Reference lists appearing in extracts for some articles
 ResolvedJdlrobsonT73023 Invalid HTML markup standalone LI element when using references on main page
 DuplicateNoneT112137 Hovercards loses manual superscript formatting by requesting plain text
 DuplicateovasilevaT152414 Previews must display useful text for the majority of articles
 ResolvedovasilevaT159388 Text Extracts used in Page Previews shows strange characters from template
 DuplicateNoneT162219 Parentheticals: Words incorrectly concatenated due to too simple removing of spaces
 DeclinedNoneT159065 Fix formulas in HTML extracts
 DuplicateJdlrobsonT163442 Math formulas are not rendering in preview page dialogs
 ResolvedMhurdT155573 [Regression] exclude pronunciation guides from article extracts
 DeclinedJMinorT164100 Consider excluding pronunciation guides from TextExtracts
 ResolvedphuedxT112226 Text extracts unable to summarise content contains {{dts}} templates
 Resolved• PcheloloT165017 Setup RESTBase HTML extract endpoint
 ResolvedJdlrobsonT165619 Spike: Specify changes to page-summary endpoint
 Declined• PcheloloT166163 Create a summary content migration filter
 ResolvedphuedxT167852 "<translate>" and <tvar|*> tag visible in Page Previews: HtmlFormatter only flattens spans
 DeclinedJdlrobsonT168329 exsentences does not work correctly when HTML output used
 DuplicateNoneT168625 Make the Page Summary API return an "intro" for a page
 Resolved• FjalapenoT167022 Inventory requirements for summary 2.0 endpoint across Reading teams
 ResolvedphuedxT168400 [EPIC] Add notion of emptiness to page summary API
 DeclinedpmiazgaT168328 Extracts for Wikimedia List articles display partial previews
 DeclinedNoneT168391 Page previews should display generic preview for disambiguation pages
 ResolvedphuedxT114418 Disable TextExtracts on file pages
 DeclinedNoneT168418 Page Previews should accept the service's notion of emptiness
 Resolved• PcheloloT164291 Make title-related properties consistent
 Resolved• bearNDT169761 Review Summary 2.0 Spec
 Resolved• MhollowayT170692 Return common URLs in summary API so clients do not have to perform bug prone string manipulation
ResolvedMSantosT177619 Return variant URLs and titles in the metadata response
 Resolved• MhollowayT178446 Expose display titles for a page in all available language variants through the action API
 InvalidNoneT69232 Hovercards: Redirect titles should be bolded in the extract
 ResolvedJdlrobsonT170617 Adjust expectations for API consumers when using the TextExtracts API
 InvalidNoneT185472 API problems with double spaces in wiki sections
 DuplicateNoneT171053 Complete entire page previews test plan for page summary api
 ResolvedJdlrobsonT171052 Add disambiguation page handling in Page Summary API
 ResolvedJdlrobsonT168848 Bootstrap an initial version of the Page Summary API in MCS
 ResolvedJdlrobsonT172021 It shouldn't be possible for coordinates to be the lead paragraph
 ResolvedJdlrobsonT174698 Parenthetical stripping is too aggressive
 ResolvedphuedxT171065 Expose disambiguation property (ppprop=disambiguation) in mobileview api
 Resolved• bearNDT175286 Do side by side comparison of old summary endpoint against new summary endpoint
 Resolved• MhollowayT176974 Mime vulnerability blocking merges to MCS
 DuplicateNoneT176063 Page preview shows only the first two words for a specific article
 ResolvedJdlrobsonT176517 Pages that are redirects give empty summary objects
 ResolvedJdlrobsonT176519 Old templates can lead to sup elements inside summary
 ResolvedphuedxT176521 data-mw attributes should be stripped from summary before scrubbing parentheticals
ResolvedJdlrobsonT176522 Intro property incorrectly identified in formatted endpoint
 ResolvedJdlrobsonT176525 Parenthetical: New edge case with nested brackets
 Resolved• bearNDT177007 Should we flatten spans in summary output
 ResolvedJdlrobsonT178125 Metadata (coordinates) is sometimes surfaced as lead intro/summary
 Resolved• MhollowayT178333 Move RESTBase page summary logic to MCS
 Resolved• MhollowayT178420 What is a "content namespace" for purposes of the summary 2.0 endpoint?
 ResolvedJdlrobsonT183833 [Bug report] Removing parentheses breaks chemical formulas
 ResolvedJdlrobsonT185050 Run comparison of html extracts again
 Resolved• bearNDT185161 Stripping parentheticals can result in trailing punctuation
 Resolved• mobrovacT179875 Update RESTBase to get summary content from MCS Summary 1.3 endpoint when development is complete
 ResolvedABorbaWMFT182399 QA html previews on beta cluster
 ResolvedABorbaWMFT182639 Page previews cannot handle empty responses gracefully - throws error in beta cluster and doesn't display preview (browser tests were failing)
 Resolved• MhollowayT186847 Initial transcluded <p> elements on dewiki are fooling extractLeadIntroduction
 Resolved• MhollowayT186933 204 summary responses cause Android app to crash
 Resolved• MhollowayT187091 Summary: stop responding with 204s
 Resolved• MhollowayT184429 Ensure "timestamp" refers to the requested revision
 DeclinedNoneT184484 Bump summary revision to 2.0
 Resolved• MhollowayT184430 Flag project main pages with type=mainpage
Restricted Task
Restricted Task
Restricted Task
Restricted Task
Restricted Task
Restricted Task
 Resolved• MhollowayT184751 Benchmark the new page summary API
 DeclinedNoneT184753 Use stored page leads when creating page summaries to reduce MCS load
 Resolved• PcheloloT186720 Etag is not returned for mobile-sections-lead or mobile-sections-remaining if a revision parameter is present
 Resolved• MhollowayT187098 Extra span at beginning of mobile-sections-lead
 InvalidNoneT162185 Extract JSON API from MCS Page Content API
 Resolved• PcheloloT189750 Expose new PCS JSON endpoints via RESTBase
 Resolved• bearNDT189751 Expose /page/media endpoint via RESTBase
 DeclinedNoneT189728 Design strategy for updating MCS/PCS content
 Resolved• bearNDT190198 Expose /page/metadata endpoint via RESTBase
 Resolved• bearNDT190337 Expose /page/references endpoint via RESTBase
 ResolvedNoneT177433 Develop HTML Content API to be used with companion structured JSON APIs
 Resolved• JhernandezT167921 Support Lazy loading of page content not needed for first paint
Resolved• bearNDT168875 Remove reference sections at end of article from page content HTML
 Resolved• bearNDT174809 Add swagger spec for content-html
 DuplicateNoneT175326 Remove navboxes from PCS HTML output
 Resolved• bearNDT170584 Evaluate and Integrate page library transforms into PCS
 Declined• bearNDT198667 Split up CollapseTable transform into client and server side code
 Resolved• bearNDT198668 Use LazyLoadTransform in content-html
 Resolved• bearNDT199491 Expose /page/mobile-html endpoint via RESTBase
 Resolved• bearNDT200459 Bug: mobile-html should use only externally visible URIs for referencing site CSS
 Resolved• bearNDT200464 mobile-html CSP issues
 Resolved• MhollowayT201380 mobile-html: image resolution too low
 Resolved• bearNDT201381 Infoboxes too narrow on wide devices
 ResolvedMhurdT202939 Integrate page library change for 6.0.1
 ResolvedDbrantT203024 Update page library version for Android app to avoid narrow infoboxes on wide devices
 Resolved• bearNDT201383 mobile-html: add page header
 Resolved• bearNDT205550 Localize the strings used by page library transforms
 Resolved• NHarateh_WMFT206856 mobile-html: iOS prototype
 ResolvedMSantosT216074 Remove navboxes from mobile-html
 ResolvedMSantosT217348 Scheme-less URLs in mobile-html
 ResolvedMSantosT218147 Allow app:// protocol in mobile-html CSP headers
 ResolvedSharvaniharanT206855 mobile-html: Android prototype
 Resolved• bearNDT217352 mobile-html page footer handling
 Resolved• JhernandezT220044 PCS CSS improvements
 Resolved• bearNDT202105 Separate pagelib CSS from base CSS
 DuplicateNoneT214171 [Bug] Evolution of the wolf article on enwiki renders incorrectly through the mobile-html endpoint on mobile safari
 Resolved• bearNDT217417 [Bug] Infobox header remains full width when infobox is only partial width on mobile-html pages
 Resolved• MhollowayT220045 Stop getting base CSS from live ResourceLoader requests
 Resolved• MhollowayT218928 Image and captions overlapping on German article in Android app
 ResolvedBUG REPORT• bearNDT217837 [BUG] mobile-html article body has wrong background color
 ResolvedBUG REPORT• JhernandezT214714 Improper CSS being served to apps.
 ResolvedBUG REPORT• MhollowayT219411 External links no longer have arrow icon in Light theme.
 ResolvedBUG REPORTDbrantT214728 Dark mode broken (page background remains light) in apps.
 ResolvedJdlrobsonT221695 [Bug] Infobox in "Iraq war" article on enwiki not full width on mobile-html
 Resolved• bearNDT223477 [Bug] enwiki LAPD article has truncated tables
 Resolved• JhernandezT222369 mobile-html: JavaScript abstraction layer
 Resolved• bearNDT219998 mobile-html interaction handling
 ResolvedMSantosT219909 mobile-html section offsets
 Resolved• bearNDT218049 mobile-html: allow clients to combine content customization tasks into a single call and include a callback (set theme, dim images, collapse tables, margins,
Resolved• bearNDT201382 mobile-html: adjust page margins/padding
 DeclinedNoneT218804 mobile-html reference handling
 Resolved• bearNDT206502 Add startup script invocation to pagelib JS output
 DeclinedNoneT218050 mobile-html: allow clients to adjust tables
 ResolvedMSantosT222752 page library: allow clients to adjust text size
 DeclinedNoneT226034 Add page image info to mobile-html
 ResolvedSpikeJMinorT228459 mobile-html: prioritized list of remaining bugs & feature requests
 ResolvedBUG REPORTMSantosT229984 [BUG] Handle footer menu icon style correctly
 ResolvedMSantosT230036 mobile-html: update footer with updated design
 ResolvedMSantosT230067 mobile-html: enhance media-list to include exact image URLs
 ResolvedJoeWalshT231201 mobile-html: improve page load time and time to first paint
 Resolved• mobrovacT232449 Expose new PCS javascript endpoints: pagelib_body_start and pagelib_body_end
 ResolvedMSantosT230040 [Bug] media-list: endpoint contains redirects
Restricted Task
 ResolvedMSantosT238916 Review and respond to /mobile/html security review
 OpenSpikeNoneT234338 Make performance test for newly deployed mobile-html
 DeclinedNoneT234843 Allow /page/mobile-html to be able to be embedded in web applications
 ResolvedMSantosT217349 mobile-html-offline-resources endpoint

Event Timeline

• Fjalapeno created this task.Oct 4 2017, 5:35 PM
• Fjalapeno created subtask T177426: Develop structured JSON APIs for general consumption.Oct 4 2017, 5:39 PM
• Fjalapeno removed a project: Services (watching).Oct 4 2017, 5:59 PM
• Fjalapeno created subtask T177433: Develop HTML Content API to be used with companion structured JSON APIs.Oct 4 2017, 6:16 PM
• Fjalapeno updated the task description. (Show Details)Oct 5 2017, 3:01 PM
• Fjalapeno updated the task description. (Show Details)
• Fjalapeno updated the task description. (Show Details)Oct 5 2017, 3:04 PM
• Fjalapeno added a project: Page Content Service.
• Fjalapeno moved this task from Needs triage to Epics on the Product-Infrastructure-Team-Backlog-Deprecated board.
• Fjalapeno updated the task description. (Show Details)Oct 5 2017, 3:29 PM
• Jhernandez triaged this task as High priority.Jul 6 2018, 12:36 PM
• Jhernandez added a subtask: T217349: mobile-html-offline-resources endpoint.Mar 13 2019, 3:48 PM
• bearND closed subtask T217349: mobile-html-offline-resources endpoint as Resolved.Jul 30 2019, 6:37 PM
MSantos moved this task from Backlog to Icebox on the Page Content Service board.Sep 2 2019, 4:47 PM
• Jhernandez closed subtask T177426: Develop structured JSON APIs for general consumption as Resolved.Sep 11 2019, 12:32 PM
MSantos closed this task as Resolved.Feb 25 2021, 4:47 PM
MSantos claimed this task.
MSantos added a subscriber: MSantos.

I will be bold and resolve the epics related to the PCS project, which is finished, stable, and now in maintenance mode. Please re-open if you think I'm missing something.

MSantos removed MSantos as the assignee of this task.Feb 25 2021, 4:47 PM
MSantos closed subtask T177433: Develop HTML Content API to be used with companion structured JSON APIs as Resolved.Feb 25 2021, 4:50 PM
Content licensed under Creative Commons Attribution-ShareAlike 3.0 (CC-BY-SA) unless otherwise noted; code licensed under GNU General Public License (GPL) or other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL