@dr0ptp4kt: Are the Reading Web team responsible for fixing this?

@phuedx, I'm not sure. @Tgr, is this a bot identifier using a bot password on this particular fishbowl wiki, and should the particular warning be suppressed centrally for successful bot password usage generally / explicitly suppressed within JCUtils.php ? Doing some archaeology, I think the account was associated with T135074: Update JsonConfig for AuthManager.

The title is a bit ambiguous - is the script actually failing to login, or is this just about making it log in in a non-deprecated manner? (It uses a bot password, you can see it from the login name, but that doesn't really make any difference.)

In T155532#2953897, @Tgr wrote:

The title is a bit ambiguous - is the script actually failing to login, or is this just about making it log in in a non-deprecated manner? (It uses a bot password, you can see it from the login name, but that doesn't really make any difference.)

It's the latter, warning about the deprecated manner which clutters my logs :)

So someone should fix JsonConfig to use action=query&meta=tokens instead of action=login to get a login token.

Following up on this, I'm opening an email thread with some stakeholders about maintenance responsibilities for JsonConfig.

Change 334024 had a related patch set uploaded (by Legoktm):
Use action=query&meta=tokens to get login token

https://gerrit.wikimedia.org/r/334024

@dr0ptp4kt who knows how to test this? If we merge and it breaks the Zero stuff on beta, will someone notice? (Is there even Zero stuff on beta?)

Best I can hope for is no complaints and errors go away in logstash. I have zero insight into how this actually works.

I can test the user-facing components, although I believe you'd also want @BBlack keeping a watch on backend scheduled jobs and the general edge. @Tgr / @Legoktm would it be possible for you to schedule a deployment window where @BBlack and I are both available?

As I recall, there is some wireup on the beta cluster for the user-facing part (i.e., ZeroBanner and JsonConfig play nicely), although I'm not aware of a Zero Portal installation that Just Works on the beta cluster. I think the deployment approach would be to (1) first push the patch such that ZeroBanner does its thing on the beta cluster from the consuming side of things so we can see that reads are healthy, and then (2) roll to "the rest of prod" where JsonConfig runs, including the ZeroPortal server, zero.wikimedia.org so we can see that both writes and reads are healthy from both the portal and ultimately end user side of things.

Does that make sense?

We can do that, but given that this issue seems to come up every year, it would be nice to solve it properly, by documenting what JsonConfig actually does and how much of that happens on beta so anyone can just change the code and check the right graphs. Currently there seems to be no wikitech documentation whatsoever on zerowiki, and the only mw.org documentation I found is Wikipedia Zero which is very generic and seems several years old.

Concerning zerowiki and ZeroPortal https://www.mediawiki.org/wiki/Extension:ZeroPortal/Conventions does cover some details as written around the time the system was written, although you're right wireup and configuration material is by and large implicit in the repos. I agree documentation and a functional non-production lower environment should at the least be examined, but I think we should hold off jumping deeply into it at the moment.

We should do the following:

1. Schedule the deployment of the fix to make the logspam go away. @Tgr, would you please arrange this at a time that works for both @BBlack and me? I think you need a full hour, as there are intentional delays built into the propagation of various pieces of data.
2. Kick off a separate thread about support for JsonConfig (I've notified some people on a separate thread, already). It's something of data access layer technology, and so it feels infrastructure-y to me - maybe something that ought to be revisited as data access abstractions are being looked it (MCR?), because at least the work would be in a similar conceptual space.
3. More generally, it's known that ZeroBanner and ZeroPortal will need their support needs reviewed, potentially including code refactor (the code's mostly been resilient, but there may be different ways to arrange it) or enhancements or both. There's a discussion forming around this already, and @DFoy and I continue to meet weekly to review issues (n.b., there is at least one additional issue that should get some attention soon). As you probably know, Reading Web supports ZeroBanner, but it has generally been pretty low key stuff - primarily maintenance for bugfixes, with some less bugfix-y items along the way, to be fair.

Change 336258 had a related patch set uploaded (by Gergő Tisza):
Use action=query&meta=tokens to get login token

https://gerrit.wikimedia.org/r/336258

Change 334024 merged by jenkins-bot:
Use action=query&meta=tokens to get login token

https://gerrit.wikimedia.org/r/334024

Change 336258 merged by Gergő Tisza:
Use action=query&meta=tokens to get login token

https://gerrit.wikimedia.org/r/336258

Mentioned in SAL (#wikimedia-operations) [2017-02-06T20:39:02Z] <tgr@tin> Synchronized php-1.29.0-wmf.10/extensions/JsonConfig/includes/JCUtils.php: T155532: Update JsonConfig login API call (duration: 01m 00s)

Follow-up: T157384: Document zerowiki setup better

Testing procedure

1. Visit https://en.m.wikipedia.org/wiki/Main_Page and ensure the internet facing non-shared IP address isn't resulting in getting zero-rating chrome.

2. Do no-op config change on zerowiki's Zero:TEST1. Confirm with @BBlack, @Tgr no confirmed issue.

3. Do config change on zerowiki's Zero:TEST1 with internet facing non-shared IP address. Confirm with @BBlack, @Tgr no observed issue.

4. Wait 15 minutes.

5. Visit https://en.m.wikipedia.org/wiki/Address_Resolution_Protocol and verify getting zero-rating chrome.

@BBlack, @Tgr, @dr0ptp4kt -

I've conducted all steps above with no issue. I've also logged out of my zero wiki account and signed back in, and was able to access and edit config pages.

Also, I've turned the test off and banners/behavior disappeared as expected within 15 minutes.