I'd like explicitly list the blockers to having the pageview api in beta. So far I see:

• beta.wmflabs.org domains not working (how? by policy or ???)
• hard to maintain the analytics cluster in beta

With that, I'd like an explicit alternative with trade-offs proposed (what Lego mentioned in his first IRC line above, I suppose), just more fleshed out.

[Oops, apparently I never submitted my comment here]

In T129602#2123685, @greg wrote:

I'd like explicitly list the blockers to having the pageview api in beta. So far I see:

• beta.wmflabs.org domains not working (how? by policy or ???)

Sorry, I meant that the *.beta.wmflabs.org domain names won't work with the production pageview API. I can implement this though.

• hard to maintain the analytics cluster in beta
  
  With that, I'd like an explicit alternative with trade-offs proposed (what Lego mentioned in his first IRC line above, I suppose), just more fleshed out.

The full IRC logs are in: http://bots.wmflabs.org/~wm-bot/logs/%23wikimedia-analytics/20160309.txt

I don't really have an answer for you, because I don't understand the costs or consequences of setting up the service in beta. I'm not sure it makes sense to block this extension on that though, mostly because I suspect that setting the service up will be entirely non-trivial, and not worth it *just* for this extension. And, this extension is a pretty low priority that even I forgot about it. -.-

In T129602#2539028, @Legoktm wrote:

I don't really have an answer for you, because I don't understand the costs or consequences of setting up the service in beta. I'm not sure it makes sense to block this extension on that though, mostly because I suspect that setting the service up will be entirely non-trivial, and not worth it *just* for this extension. And, this extension is a pretty low priority that even I forgot about it. -.-

Yeah, that's not incorrect :)

"We" should still figure out why the analytics cluster in beta cluster is so hard to maintain.

06:13 < anomie> tgr|away, legoktm, bd808: PageViewInfo could take the same tack as ORES: ORES has a "testing" wiki that scores any revision as a transform of the rev_id (e.g. rev_id 123456 is scored 0.65), which was good enough for me to be able to do the development needed for T143895.

I agree with that; getting analytics set of for beta would be cool, but no reason to block this task on that. Just set up a fake service that generates pageview stats from the article title in some deterministic way and uses the same response format as the real one. (If the "pretend we are on a production wiki" approach does not work, anyway. That would be even simpler, and it's a public API so I don't see any disadvantage.)

I feel like we're heading into NFS-vs.-Swift-like issues again with this 'fake' service, but okay...

In T129602#2732374, @Tgr wrote:

(If the "pretend we are on a production wiki" approach does not work, anyway. That would be even simpler, and it's a public API so I don't see any disadvantage.)

I think we're all in agreement that we should not let the lack of a beta cluster page view API end-point block the deployment of this extension to Wikimedia wikis.

Options I see:

1. Fake end-point as described by Tgr.
2. Point to production end-point on beta wikis until there's an alternative to use.
3. Bypass beta wikis and deploy to phase 0 wikis (test.wikipedia.org and friends).

I like options 2 and 3 better than option 1. @greg and others: any thoughts?

It would defeat the point of beta (even if that point is a somewhat indistinct mix of testing, integration and staging) if we would deploy extensions to production without deploying them there first. I agree 2 would be better than 1 (that's how I plan to set up the vagrant role, too).

We have CheckUser in production but not beta.

In T129602#2733358, @Tgr wrote:

It would defeat the point of beta (even if that point is a somewhat indistinct mix of testing, integration and staging) if we would deploy extensions to production without deploying them there first.

I'm not going to argue about the point, purpose, or virtue of having the beta wikis. However I will note that dozens of extensions have been deployed to Wikimedia wikis without first going to the beta wikis and wikis such as test.wikipedia.org far pre-date the existence of the beta wikis.

If option 2 sounds good, I think we should do that and then we can mark this task resolved. :-) Reedy and/or Addshore seemed vaguely interested in helping get this extension deployed, but if you and Anomie (and Krenair?) are also interested in helping out, that would be great too! I'm really looking forward to being able to incorporate page view data into the user interface.

In T129602#2733581, @MZMcBride wrote:

In T129602#2733358, @Tgr wrote:

It would defeat the point of beta (even if that point is a somewhat indistinct mix of testing, integration and staging) if we would deploy extensions to production without deploying them there first.

I'm not going to argue about the point, purpose, or virtue of having the beta wikis. However I will note that dozens of extensions have been deployed to Wikimedia wikis without first going to the beta wikis and wikis such as test.wikipedia.org far pre-date the existence of the beta wikis.

Processes change :) To get a new extension into production, we require testing in Beta Cluster.

Given I still haven't seen a comment from Analytics that wasn't made via a third party, I still want to know why they can't set this service up in Beta Cluster now. "It's hard" is not a valid (single) reason. Much of your/our work is hard. We've had outages in production with the lack of real testing of ORES in Beta Cluster already. See, eg: https://wikitech.wikimedia.org/wiki/Incident_documentation/20160801-ORES

(for the record, the answer still could be something like MZ's #2, but I'm not going to blindly go there without concrete reasons we're skipping best practices.)

In T129602#2739924, @greg wrote:

Given I still haven't seen a comment from Analytics that wasn't made via a third party, I still want to know why they can't set this service up in Beta Cluster now.

If you or @Tgr are interested in getting a service set up on the beta cluster, that sounds like a separate task.

If you're trying to get a comment from a specific person from the Wikimedia Foundation analytics team, I recommend e-mailing the person directly.

"It's hard" is not a valid (single) reason. Much of your/our work is hard.

No it isn't.

Deploying this extension to the beta cluster should be fairly straightforward.

In T129602#2741033, @MZMcBride wrote:

Deploying this extension to the beta cluster should be fairly straightforward.

Right, the point of the discussion at hand, however, is: what benefit will we get from that deployment. Which is why I asked for a comment from the analytics team about setting up the pageview api in Beta.

Analytics team: Can you see this task and respond to https://phabricator.wikimedia.org/T129602#2739924, please? Thank you.

@greg:(sorry I did not see this ticket until today)

There are two topics here:

1. Setting up an analytics stack in beta
2. Setting up a test PageviewAPI endpoint

Regarding 1) :

I was keen on having an analytics stack in beta until we realized that (unlike the mediawiki stack) an analytics stack in beta would not give us any level of assurance regarding whether things work in production. On our end problems are apparent when we deal with huge amounts of data (in size and variety) and beta hardware couldn't even sustain the moderate stream of pageviews we have in beta env. Our prior attempts to setup the pageview pipeline just died due to resource constrains (cc @Ottomata).

So, I cannot really see a value proposition of us maintaining an analytics stack in beta if it doesn't help our testing. (I guess in theory we could test upgrades of software but I am not even sure if that would be the case). I think the best value that a beta analytics environment would provide would be user training, but that is far off from beta's purpose which is true integration testing.

Regarding 2)
The PageviewAPI feeds of the analytics pipeline in prod but what clients see is an isolated http entry point. I think we could deploy a mock version of it to beta. I have worked in several SOA testing environments in which a similar approach was used and that was of value.
On our end we know we could benefit from a testing stack for pageview API better than the one we have.

Now, if this extension needs to be deployed ASAP then deploy to phase 0 wikis is the way to go.

In T129602#2758048, @Nuria wrote:

Now, if this extension needs to be deployed ASAP then deploy to phase 0 wikis is the way to go.

There's no major urgency, just a desire to help our users. :-)

Don't want to do this until T148775 is fixed, but I'm happy to deploy this to beta and/or production afterwards as deemed appropriate

Change 320333 had a related patch set uploaded (by Reedy):
Add PageViewInfo to beta

https://gerrit.wikimedia.org/r/320333

Change 320333 merged by jenkins-bot:
Add PageViewInfo to beta

https://gerrit.wikimedia.org/r/320333

Change 320450 had a related patch set uploaded (by Gergő Tisza):
Make beta PageViewInfo use the production pageview API

https://gerrit.wikimedia.org/r/320450

Change 320552 had a related patch set uploaded (by Gergő Tisza):
Add PageViewInfo log channel

https://gerrit.wikimedia.org/r/320552

Change 320450 merged by jenkins-bot:
Make beta PageViewInfo use the production pageview API

https://gerrit.wikimedia.org/r/320450

Change 320552 merged by jenkins-bot:
Add PageViewInfo log channel

https://gerrit.wikimedia.org/r/320552

Mentioned in SAL (#wikimedia-operations) [2016-11-10T19:09:52Z] <thcipriani@tin> Synchronized wmf-config/InitialiseSettings-labs.php: SWAT: [[gerrit:320450|Make beta PageViewInfo use the production pageview API (T129602)]] (labs-only-change) (duration: 00m 48s)

Mentioned in SAL (#wikimedia-operations) [2016-11-10T19:16:53Z] <thcipriani@tin> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:320552|Add PageViewInfo log channel (T129602)]] (duration: 00m 49s)