Page MenuHomePhabricator
Create Task
Maniphest T349517

Ensure Wikifeeds requests intended for AQS 1.0 are correctly sent to AQS 2.0
Closed, ResolvedPublic5 Estimated Story Points
Actions

Description

Description

After the Pageview Service migrated from AQS 1.0 to 2.0 we noticed that there remained requests going to AQS 1.0. There are two origins for these requests, this ticket concerns the requests coming from Wikifeeds.

Wikifeeds requests pageviews here and will require addition of an extra configuration option to specify using the rest-gateway for these requests as opposed to restbase (which is used elsewhere).

The relevant config is here.

Acceptance Criteria

  • It is possible to separately configure requests bound for AQS vs requests bound for restbase

Required

  • Unit/Integration tests
  • Documentation
  • Passed QA

Details

SubjectRepoBranchLines +/-
wikifeeds: add rest-gateway config and bump imageoperations/deployment-chartsmaster+12 -1
rest-gateway: respond on service mesh hostname:portoperations/deployment-chartsmaster+4 -0
wikifeeds: configure log leveloperations/deployment-chartsmaster+12 -3
rest-gateway: change how AQS URLs enforce wikimedia.org domainoperations/deployment-chartsmaster+14 -20
Make pageviews API requests via REST Gatewaymediawiki/services/wikifeedsmaster+125 -27
Customize query in gerrit

Related Objects
Search...

Event Timeline

WDoranWMF triaged this task as High priority.Oct 23 2023, 2:25 PM
WDoranWMF created this task.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 23 2023, 2:25 PM
WDoranWMF added a parent task: T348731: Follow up on remaining requests to pageviews endpoints .Oct 23 2023, 2:25 PM
MSantos added a project: Content-Transform-Team.Oct 24 2023, 10:50 AM
MSantos moved this task from Backlog to Tracking on the Content-Transform-Team board.
MSantos added subscribers: Jgiannelos, MSantos.
BTullis subscribed.Oct 24 2023, 1:56 PM
gerritbot added a comment.Oct 30 2023, 5:14 PM

Change 969933 had a related patch set uploaded (by Phuedx; author: Phuedx):

[mediawiki/services/wikifeeds@master] WIP: Request pageviews from API Gateway

https://gerrit.wikimedia.org/r/969933

gerritbot added a project: Patch-For-Review.Oct 30 2023, 5:14 PM
phuedx subscribed.EditedOct 30 2023, 5:23 PM

☝️ WIP because:

  1. I've incorrectly said API Gateway when I should have said REST Gateway…
  2. I need help setting the production value for the new configuration variable – what's the base URI for the pageviews endpoint(s) on the REST Gateway?
  3. There's an integration test that's failing on both the master branch and my work branch and it involves a magic number
phuedx claimed this task.Oct 30 2023, 5:24 PM
phuedx moved this task from Sprint Backlog to In Process on the Data Products (Data Products (Sprint 03)) board.
phuedx added a subscriber: Milimetric.
phuedx added a comment.Oct 31 2023, 11:35 AM
In T349517#9292513, @phuedx wrote:

☝️ WIP because:

  1. I've incorrectly said API Gateway when I should have said REST Gateway…
  2. I need help setting the production value for the new configuration variable – what's the base URI for the pageviews endpoint(s) on the REST Gateway?
  3. There's an integration test that's failing on both the master branch and my work branch and it involves a magic number

Update: 1 and 2 are done. 3 is ongoing.

VirginiaPoundstone lowered the priority of this task from High to Low.Oct 31 2023, 12:18 PM
phuedx moved this task from In Process to In code review / Tech Input on the Data Products (Data Products (Sprint 03)) board.Oct 31 2023, 12:19 PM
gerritbot added a comment.Nov 3 2023, 11:25 AM

Change 969933 merged by jenkins-bot:

[mediawiki/services/wikifeeds@master] Make pageviews API requests via REST Gateway

https://gerrit.wikimedia.org/r/969933

phuedx mentioned this in rWSWF71a8901555a1: Make pageviews API requests via REST Gateway.Nov 3 2023, 12:04 PM
gerritbot added a comment.Nov 3 2023, 12:04 PM

Change 971407 had a related patch set uploaded (by Hnowlan; author: Hnowlan):

[operations/deployment-charts@master] wikifeeds: bump image

https://gerrit.wikimedia.org/r/971407

hnowlan subscribed.Nov 3 2023, 12:27 PM

Minor hitch with our change - currently the rest-gateway requires the Host header to be set as wikimedia.org to avoid cache pollution. However, wikifeeds uses analytics.wikimedia.org. I'd like to relax the host header requirement, but enforce this via the URL path instead - so instead using wikimedia.org instead of analytics.wikimedia.org in the URLs requesting pageviews data. Where does this value come from in wikifeeds? I cant see it in the repo or the helm chart

hnowlan added a comment.Nov 3 2023, 1:24 PM
In T349517#9304785, @hnowlan wrote:

Minor hitch with our change - currently the rest-gateway requires the Host header to be set as wikimedia.org to avoid cache pollution. However, wikifeeds uses analytics.wikimedia.org. I'd like to relax the host header requirement, but enforce this via the URL path instead - so instead using wikimedia.org instead of analytics.wikimedia.org in the URLs requesting pageviews data. Where does this value come from in wikifeeds? I cant see it in the repo or the helm chart

Never mind, this is incorrect reading on my part!

gerritbot added a comment.Nov 3 2023, 1:33 PM

Change 971456 had a related patch set uploaded (by Hnowlan; author: Hnowlan):

[operations/deployment-charts@master] rest-gateway: change how AQS URLs enforce wikimedia.org domain

https://gerrit.wikimedia.org/r/971456

phuedx moved this task from In code review / Tech Input to To Deploy on the Data Products (Data Products (Sprint 03)) board.Nov 3 2023, 3:33 PM
gerritbot added a comment.Nov 6 2023, 12:31 PM

Change 971456 merged by jenkins-bot:

[operations/deployment-charts@master] rest-gateway: change how AQS URLs enforce wikimedia.org domain

https://gerrit.wikimedia.org/r/971456

gerritbot added a comment.Nov 7 2023, 11:51 AM

Change 971407 merged by jenkins-bot:

[operations/deployment-charts@master] wikifeeds: add rest-gateway config and bump image

https://gerrit.wikimedia.org/r/971407

gerritbot added a comment.Nov 7 2023, 12:28 PM

Change 972362 had a related patch set uploaded (by Hnowlan; author: Hnowlan):

[operations/deployment-charts@master] wikifeeds: configure log level

https://gerrit.wikimedia.org/r/972362

gerritbot added a comment.Nov 7 2023, 2:41 PM

Change 972362 merged by jenkins-bot:

[operations/deployment-charts@master] wikifeeds: configure log level

https://gerrit.wikimedia.org/r/972362

gerritbot added a comment.Nov 7 2023, 5:18 PM

Change 972427 had a related patch set uploaded (by Hnowlan; author: Hnowlan):

[operations/deployment-charts@master] rest-gateway: respond on service mesh hostname:port

https://gerrit.wikimedia.org/r/972427

gerritbot added a comment.Nov 7 2023, 5:50 PM

Change 972427 merged by jenkins-bot:

[operations/deployment-charts@master] rest-gateway: respond on service mesh hostname:port

https://gerrit.wikimedia.org/r/972427

phuedx reassigned this task from phuedx to hnowlan.Nov 8 2023, 11:42 AM

☝️ Reflecting reality…

phuedx moved this task from To Deploy to Done on the Data Products (Data Products (Sprint 03)) board.Nov 8 2023, 11:43 AM
hnowlan added a comment.Nov 9 2023, 5:50 PM

I think we're all good here - can't fully verify until we return public traffic to page-analytics, but we can probably resolve and reopen if needs be.

WDoranWMF closed this task as Resolved.Nov 20 2023, 5:44 PM
Maintenance_bot removed a project: Patch-For-Review.Nov 20 2023, 6:12 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL