Page MenuHomePhabricator

Deploy WikimediaPageViewInfo extension to beta cluster
Closed, ResolvedPublic

Description

Before it goes into production, etc.

[16:57:03] <legoktm> greg-g: so I'm figuring out how to set up the WikimediaPageViewInfo extension on beta cluster...problem is that there is no page view api in beta. So I was thinking it could just point at the production one for now? Also there's the issue that the *.beta.wmflabs.org domains won't work, so in -analytics we had discussed just picking some medium wiki and pretending all of beta was that wiki. Or something.
[16:58:18] <greg-g> legoktm: hrmmmm
[16:58:41] <greg-g> I uh, need to think about that, is there a reason we can't get the pageveiewinfo for betacluster?
[16:59:32] <legoktm> [11:31:41] <madhuvishy> legoktm: no - we have a beta cluster version of the analytics cluster - but too much of a pain to maintain so far and the instances have been dying. so its even harder to get the full pipeline with cassandra setup
[16:59:37] <legoktm> from -analytics yesterday
[17:00:09] <greg-g> :(
[17:00:53] -*- greg-g goes afk will follow up tomorrow or so
[17:01:15] <legoktm> I'll file a task for it

Event Timeline

Legoktm created this task.Mar 11 2016, 1:02 AM
greg updated the task description. (Show Details)Mar 11 2016, 6:25 PM
greg added a comment.Mar 15 2016, 6:56 PM

I'd like explicitly list the blockers to having the pageview api in beta. So far I see:

  • beta.wmflabs.org domains not working (how? by policy or ???)
  • hard to maintain the analytics cluster in beta

With that, I'd like an explicit alternative with trade-offs proposed (what Lego mentioned in his first IRC line above, I suppose), just more fleshed out.

[Oops, apparently I never submitted my comment here]

I'd like explicitly list the blockers to having the pageview api in beta. So far I see:

  • beta.wmflabs.org domains not working (how? by policy or ???)

Sorry, I meant that the *.beta.wmflabs.org domain names won't work with the production pageview API. I can implement this though.

  • hard to maintain the analytics cluster in beta

With that, I'd like an explicit alternative with trade-offs proposed (what Lego mentioned in his first IRC line above, I suppose), just more fleshed out.

The full IRC logs are in: http://bots.wmflabs.org/~wm-bot/logs/%23wikimedia-analytics/20160309.txt

I don't really have an answer for you, because I don't understand the costs or consequences of setting up the service in beta. I'm not sure it makes sense to block this extension on that though, mostly because I suspect that setting the service up will be entirely non-trivial, and not worth it *just* for this extension. And, this extension is a pretty low priority that even I forgot about it. -.-

greg added a comment.Aug 10 2016, 4:15 PM

I don't really have an answer for you, because I don't understand the costs or consequences of setting up the service in beta. I'm not sure it makes sense to block this extension on that though, mostly because I suspect that setting the service up will be entirely non-trivial, and not worth it *just* for this extension. And, this extension is a pretty low priority that even I forgot about it. -.-

Yeah, that's not incorrect :)

"We" should still figure out why the analytics cluster in beta cluster is so hard to maintain.

Tgr added a subscriber: Tgr.EditedOct 20 2016, 6:03 PM
06:13 < anomie> tgr|away, legoktm, bd808: PageViewInfo could take the same tack as ORES: ORES has a "testing" wiki that scores any revision as a transform of the rev_id (e.g. rev_id 123456 is scored 0.65), which was good enough for me to be able to do the development needed for T143895.

I agree with that; getting analytics set of for beta would be cool, but no reason to block this task on that. Just set up a fake service that generates pageview stats from the article title in some deterministic way and uses the same response format as the real one. (If the "pretend we are on a production wiki" approach does not work, anyway. That would be even simpler, and it's a public API so I don't see any disadvantage.)

I feel like we're heading into NFS-vs.-Swift-like issues again with this 'fake' service, but okay...

(If the "pretend we are on a production wiki" approach does not work, anyway. That would be even simpler, and it's a public API so I don't see any disadvantage.)

I think we're all in agreement that we should not let the lack of a beta cluster page view API end-point block the deployment of this extension to Wikimedia wikis.

Options I see:

  1. Fake end-point as described by Tgr.
  2. Point to production end-point on beta wikis until there's an alternative to use.
  3. Bypass beta wikis and deploy to phase 0 wikis (test.wikipedia.org and friends).

I like options 2 and 3 better than option 1. @greg and others: any thoughts?

Tgr added a comment.Oct 20 2016, 11:00 PM

It would defeat the point of beta (even if that point is a somewhat indistinct mix of testing, integration and staging) if we would deploy extensions to production without deploying them there first. I agree 2 would be better than 1 (that's how I plan to set up the vagrant role, too).

We have CheckUser in production but not beta.

It would defeat the point of beta (even if that point is a somewhat indistinct mix of testing, integration and staging) if we would deploy extensions to production without deploying them there first.

I'm not going to argue about the point, purpose, or virtue of having the beta wikis. However I will note that dozens of extensions have been deployed to Wikimedia wikis without first going to the beta wikis and wikis such as test.wikipedia.org far pre-date the existence of the beta wikis.

If option 2 sounds good, I think we should do that and then we can mark this task resolved. :-) Reedy and/or Addshore seemed vaguely interested in helping get this extension deployed, but if you and Anomie (and Krenair?) are also interested in helping out, that would be great too! I'm really looking forward to being able to incorporate page view data into the user interface.

greg added a comment.Oct 24 2016, 9:19 PM

It would defeat the point of beta (even if that point is a somewhat indistinct mix of testing, integration and staging) if we would deploy extensions to production without deploying them there first.

I'm not going to argue about the point, purpose, or virtue of having the beta wikis. However I will note that dozens of extensions have been deployed to Wikimedia wikis without first going to the beta wikis and wikis such as test.wikipedia.org far pre-date the existence of the beta wikis.

Processes change :) To get a new extension into production, we require testing in Beta Cluster.

Given I still haven't seen a comment from Analytics that wasn't made via a third party, I still want to know why they can't set this service up in Beta Cluster now. "It's hard" is not a valid (single) reason. Much of your/our work is hard. We've had outages in production with the lack of real testing of ORES in Beta Cluster already. See, eg: https://wikitech.wikimedia.org/wiki/Incident_documentation/20160801-ORES

(for the record, the answer still could be something like MZ's #2, but I'm not going to blindly go there without concrete reasons we're skipping best practices.)

Given I still haven't seen a comment from Analytics that wasn't made via a third party, I still want to know why they can't set this service up in Beta Cluster now.

If you or @Tgr are interested in getting a service set up on the beta cluster, that sounds like a separate task.

If you're trying to get a comment from a specific person from the Wikimedia Foundation analytics team, I recommend e-mailing the person directly.

"It's hard" is not a valid (single) reason. Much of your/our work is hard.

No it isn't.

Deploying this extension to the beta cluster should be fairly straightforward.

Deploying this extension to the beta cluster should be fairly straightforward.

Right, the point of the discussion at hand, however, is: what benefit will we get from that deployment. Which is why I asked for a comment from the analytics team about setting up the pageview api in Beta.

Analytics team: Can you see this task and respond to https://phabricator.wikimedia.org/T129602#2739924, please? Thank you.

Nuria moved this task from Incoming to Radar on the Analytics board.Oct 31 2016, 3:35 PM

@greg:(sorry I did not see this ticket until today)

There are two topics here:

  1. Setting up an analytics stack in beta
  2. Setting up a test PageviewAPI endpoint

Regarding 1) :

I was keen on having an analytics stack in beta until we realized that (unlike the mediawiki stack) an analytics stack in beta would not give us any level of assurance regarding whether things work in production. On our end problems are apparent when we deal with huge amounts of data (in size and variety) and beta hardware couldn't even sustain the moderate stream of pageviews we have in beta env. Our prior attempts to setup the pageview pipeline just died due to resource constrains (cc @Ottomata).

So, I cannot really see a value proposition of us maintaining an analytics stack in beta if it doesn't help our testing. (I guess in theory we could test upgrades of software but I am not even sure if that would be the case). I think the best value that a beta analytics environment would provide would be user training, but that is far off from beta's purpose which is true integration testing.

Regarding 2)
The PageviewAPI feeds of the analytics pipeline in prod but what clients see is an isolated http entry point. I think we could deploy a mock version of it to beta. I have worked in several SOA testing environments in which a similar approach was used and that was of value.
On our end we know we could benefit from a testing stack for pageview API better than the one we have.

Now, if this extension needs to be deployed ASAP then deploy to phase 0 wikis is the way to go.

Now, if this extension needs to be deployed ASAP then deploy to phase 0 wikis is the way to go.

There's no major urgency, just a desire to help our users. :-)

Reedy added a subscriber: Reedy.Nov 1 2016, 5:00 PM

Don't want to do this until T148775 is fixed, but I'm happy to deploy this to beta and/or production afterwards as deemed appropriate

Change 320333 had a related patch set uploaded (by Reedy):
Add PageViewInfo to beta

https://gerrit.wikimedia.org/r/320333

Change 320333 merged by jenkins-bot:
Add PageViewInfo to beta

https://gerrit.wikimedia.org/r/320333

Change 320450 had a related patch set uploaded (by Gergő Tisza):
Make beta PageViewInfo use the production pageview API

https://gerrit.wikimedia.org/r/320450

Change 320552 had a related patch set uploaded (by Gergő Tisza):
Add PageViewInfo log channel

https://gerrit.wikimedia.org/r/320552

Change 320450 merged by jenkins-bot:
Make beta PageViewInfo use the production pageview API

https://gerrit.wikimedia.org/r/320450

Change 320552 merged by jenkins-bot:
Add PageViewInfo log channel

https://gerrit.wikimedia.org/r/320552

Mentioned in SAL (#wikimedia-operations) [2016-11-10T19:09:52Z] <thcipriani@tin> Synchronized wmf-config/InitialiseSettings-labs.php: SWAT: [[gerrit:320450|Make beta PageViewInfo use the production pageview API (T129602)]] (labs-only-change) (duration: 00m 48s)

Mentioned in SAL (#wikimedia-operations) [2016-11-10T19:16:53Z] <thcipriani@tin> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:320552|Add PageViewInfo log channel (T129602)]] (duration: 00m 49s)

Tgr closed this task as Resolved.Nov 10 2016, 8:10 PM
Tgr claimed this task.
Tgr removed a project: Patch-For-Review.

The extension has been deployed; I split the part about setting up a mock Pageview API endpoint to T150483: Set up a fake Pageview API endpoint for the beta cluster. The extension now uses enwiki data, which is fine for testing the extension but not useful for testing the API (ie. detecting that a change to the Pageview API code would break the extension) so it would be nice to get a mock service eventually. (Also the extension now has an awkward smoke test which requests data from enwiki and tries to guess whether that data is right; a deterministic mock API where you can fix the expected return value in tests would be helpful.)