Page MenuHomePhabricator

All users autoblocked from editing on Beta Wikis (IP address detected incorrectly as the deployment-cache-text06 internal IP)
Closed, ResolvedPublicBUG REPORT

Description

Steps to Reproduce: Navigate to https://commons.wikimedia.beta.wmflabs.org/wiki/Special:Upload

Expected Results: Upload form appears and workflow proceeds as usual

Actual Results: Autoblocked message appears. The provided username and IP associated with the block are 172.16.1.181. (In fact, I am logged in as Mdholloway and connecting from a public IP address.)

Confirmed on enwp beta as detecting wrong IP as well

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 13 2020, 2:39 PM
Mholloway updated the task description. (Show Details)Apr 13 2020, 2:40 PM

Can reproduce - same error shown, same IP detected for me as well

RhinosF1 added a subscriber: RhinosF1.EditedApr 13 2020, 5:32 PM

172.16.1.181 should never show as connecting to public internet. It’s part of the BOGON IPs.

bd808 added a subscriber: bd808.Apr 13 2020, 5:51 PM

172.16.1.181 should never show as connecting to public internet. It’s part of the BOGON IPs.

172.16.0.0/21 is the internal IP space for instances in the Cloud VPS network. 172.16.1.181 is the Cloud VPS facing ip address of the deployment-cache-text06.deployment-prep.eqiad1.wikimedia.cloud instance. That instance is part of the "CDN" layer for deployment-prep (aka "beta cluster").

I don't remember if we are or are not explicitly keeping the MediaWiki servers from seeing the x-forwared-for data in deployment-prep or not.

The auto block has expired and as such commons beta is editable but the issue will probably re-occur as soon as another block is placed assuming the fix to detect your actual IP isn’ applied.

Mholloway renamed this task from Cannot edit on Beta Commons (user detection bug?) to Cannot edit on Beta Commons (IP address detected incorrectly as the deployment-cache-text06 internal IP).Apr 14 2020, 1:35 PM
Mholloway triaged this task as High priority.Apr 14 2020, 1:37 PM

the issue will probably re-occur as soon as another block is placed assuming the fix to detect your actual IP isn’ applied.

It does.

I'm going to be bold and call this high-priority.

Mholloway renamed this task from Cannot edit on Beta Commons (IP address detected incorrectly as the deployment-cache-text06 internal IP) to All users autoblocked from editing on Beta Commons (IP address detected incorrectly as the deployment-cache-text06 internal IP).Apr 14 2020, 1:37 PM
RhinosF1 added a comment.EditedApr 14 2020, 1:43 PM

the issue will probably re-occur as soon as another block is placed assuming the fix to detect your actual IP isn’ applied.

It does.

Read: It absolutely will occur again (as it has done) until the actual IP is detected and not the internal one.

I'm going to be bold and call this high-priority.

It probably is, takes down at least two beta wikis (possibly all), It affects enwp beta + commons beta at least as well from my quick test.

RhinosF1 renamed this task from All users autoblocked from editing on Beta Commons (IP address detected incorrectly as the deployment-cache-text06 internal IP) to All users autoblocked from editing on Beta Wikis (IP address detected incorrectly as the deployment-cache-text06 internal IP).Apr 14 2020, 1:43 PM
RhinosF1 updated the task description. (Show Details)
hashar claimed this task.EditedApr 14 2020, 2:06 PM
hashar added a subscriber: hashar.

Caused by

2020-04-11

15:07 Migrated from deployment-cache-text05 (stretch) to deployment-cache-text06 (buster) - class stopped working on stretch with https://gerrit.wikimedia.org/r/c/operations/puppet/+/584553 - shut down old instance - T250006

Change 588706 had a related patch set uploaded (by Hashar; owner: Hashar):
[operations/mediawiki-config@master] beta: update text cache IP

https://gerrit.wikimedia.org/r/588706

Change 588706 merged by jenkins-bot:
[operations/mediawiki-config@master] beta: update text cache IP

https://gerrit.wikimedia.org/r/588706

The config change should have been deployed on beta now ( https://gerrit.wikimedia.org/r/#/c/operations/mediawiki-config/+/588706/ ). So tentatively that should be fixed?

Mholloway closed this task as Resolved.Apr 14 2020, 2:34 PM

Fixed! Thank you, @hashar!

antoine-approve

comment best seen in the web interface