Page MenuHomePhabricator

Beta cluster IP block page should not point to noc@wikimedia.org
Closed, ResolvedPublic

Description

Currently the beta cluster "your IP has been blocked" page tells users to contact noc@wikimedia.org to be unblocked. That should point to some other email address instead or that sentence should be removed entirely.

Details

Other Assignee
BCornwall
Related Changes in Gerrit:

Event Timeline

This bit in operations/puppet.git:modules/varnish/templates/wikimedia-frontend.vcl.erb is what would need to change:

// Block requests from IPs in blocked_nets. It is important to do this
// early but after recv_fe_ip_processing has been called, as the procedure
// takes care of writing X-Client-IP if it the request did not come
// through the TLS terminator
if (std.ip(req.http.X-Client-IP, "192.0.2.1") ~ blocked_nets) {
    return (synth(403, "Requests from your IP have been blocked, please contact noc@wikimedia.org"));
}

One approach would be to move the existing message into a hiera variable and then override that setting in the deployment-prep deployment.

I don't think we really have a shared email account that is ideal for handling end user issues like this. I wonder if we could point at a page under https://wikitech.wikimedia.org/wiki/Nova_Resource:Deployment-prep where we could explain the likely problem (caught in a large network block we banned to slow down crawlers) and give instructions on creating a Phab task requesting a hole in the block?

There are three more 4xx synthetic responses in the same area of the Varnish config. They are all related to user-agent blocks of various kinds and all seem to return the same message: Scripted requests from your IP have been blocked, please see https://meta.wikimedia.org/wiki/User-Agent_policy. In case of further questions, please contact noc@wikimedia.org.

A handful of other noc@wikimedia.org references are present in other vcl files in the ::varnish module.

We got another email to noc@ about this. Is there at least an address it can be forwarded to?

We got another email to noc@ about this. Is there at least an address it can be forwarded to?

For lack of anything better you can bounce them to my work address. bd808-as-a-service doesn't scale, but maybe I'm better than a blackhole route.

Change #1143602 had a related patch set uploaded (by Majavah; author: Majavah):

[operations/puppet@production] varnish: Allow customising "contact noc@" error

https://gerrit.wikimedia.org/r/1143602

I have created https://wikitech.wikimedia.org/wiki/Nova_Resource:Deployment-prep/Blocked_help and a redirect at https://wikitech.wikimedia.org/wiki/Beta/Blocked that we can use to tell folks how to work around the various error responses they may be getting.

Mentioned in SAL (#wikimedia-releng) [2025-05-22T21:09:11Z] <bd808> Added block_help: "see https://wikitech.wikimedia.org/wiki/Beta/Blocked_help for more information." under profile::cache::varnish::frontend::fe_vcl_config in both deployment-cache-text and deployment-cache-upload Prefix Puppet (T393404)

Mentioned in SAL (#wikimedia-releng) [2025-05-22T21:12:53Z] <bd808> Forced Puppet run and restarted varnins-frontend on deployment-cache-text08 to pick up new config (T393404)

Mentioned in SAL (#wikimedia-releng) [2025-05-22T21:15:52Z] <bd808> Forced Puppet run and restarted varnins-frontend on deployment-cache-upload08 to pick up new config (T393404)

With the cherry-pick in place and the hiera customization the block screen looks something like this now:

Screenshot 2025-05-30 at 22.07.38.png (1×1 px, 72 KB)

Change #1143602 merged by BCornwall:

[operations/puppet@production] varnish: Allow customising "contact noc@" error

https://gerrit.wikimedia.org/r/1143602

bd808 updated Other Assignee, added: BCornwall.