One example of this: the Googlebot IP space, plus also, other Google-owned IP space that is neither Googlebot nor Google Cloud customers.
Description
Description
Details
Details
Customize query in gerrit
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Restricted Task | |||||
| Open | None | T305580 requestctl v1 improvements | |||
| Open | RLazarus | T305581 ipblocks support for other "entities" (not clouds, not abuse nets) |
Event Timeline
Comment Actions
Change 777899 had a related patch set uploaded (by RLazarus; author: RLazarus):
[operations/puppet@production] external_clouds_vendors: Support entity types besides "cloud"
Comment Actions
Ipblock, per se, supports arbitrary scope names.
What we need is to add support for thes other scopes in VCL.
My proposal would be to ditch the X-Public-Cloud header going forward, and add a X-SRE header (or whatever other name we want to give to it) and collect all these info as tags in it. So for example, a request from an IP in google cloud will have X-SRE: cloud=gcp as a fake-header, while a request coming from a known crawler IP space will have possibly a combined value, like X-SRE: cloud=aws;crawler=wikiband.
This has the disadvantage of making our VCL clunkier (every new category will need the same piece of code handling appending values to a header) but is future-proof and gives us a lot more flexibility.
Comment Actions
I suggest something simpler:
Use a common prefix in the header name, with the name of the ipblock group as the suffix.
X-SRE-Ipblock-Cloud
X-SRE-Ipblock-CrawlerEntity
etc etc
Comment Actions
Change 777899 merged by RLazarus:
[operations/puppet@production] external_clouds_vendors: Support entity types besides "cloud"
Comment Actions
Change 779149 had a related patch set uploaded (by RLazarus; author: RLazarus):
[operations/puppet@production] external_clouds_vendors: Remove migration shim for T305581
Comment Actions
Change 779157 had a related patch set uploaded (by RLazarus; author: RLazarus):
[operations/puppet@production] external_cloud_vendors: Add a known-clients/Googlebot ipblock
Comment Actions
Change 779444 had a related patch set uploaded (by Volans; author: Volans):
[operations/puppet@production] cloud vendors: force yaml output format
Comment Actions
Change 779149 merged by RLazarus:
[operations/puppet@production] external_clouds_vendors: Remove migration shim for T305581
Comment Actions
Change 779444 merged by Volans:
[operations/puppet@production] cloud vendors: force yaml output format
Comment Actions
Change 779157 merged by RLazarus:
[operations/puppet@production] external_cloud_vendors: Add a known-clients/Googlebot ipblock
Comment Actions
@RLazarus were you going to work on the rest of this? We still need more plumbing inside requestctl correct?
Comment Actions
Yeah -- I can do the implementation but I'm not sure if we've settled on what we want it to look like.
I don't have a strong opinion between the two header proposals above, did you and @Joe settle on a preference?
Comment Actions
Change 784761 had a related patch set uploaded (by RLazarus; author: RLazarus):
[operations/puppet@production] varnish: Rename public_clouds.json to ipblock_cloud.json
Comment Actions
Change 784774 had a related patch set uploaded (by RLazarus; author: RLazarus):
[operations/puppet@production] varnish: Allow using netmapper with multiple requestctl ipblocks
Comment Actions
Change 784798 had a related patch set uploaded (by RLazarus; author: RLazarus):
[operations/puppet@production] cache: Support multiple requestctl ipblock types in netmapper confd template