Page MenuHomePhabricator
Create Task
Maniphest T422249

Generate all DSL that we create out of HP in HP
Closed, ResolvedPublic
Actions

Description

Currently, there's two types of DSL used for HP that isn't generated directly by HP:

  • The haproxy map output of ipblocks
  • the rate-limiting stanzas for known-clients

we should generate them directly in HP instead of relying on confd templates.

Details

Related Changes in Gerrit:
SubjectAuthorRepoBranchLines +/-
cache::varnish: switch known client rate limits to hp-generated dataGiuseppe Lavagettooperations/puppetproduction+2 -35
cache::varnish: add rate-limit file generated from hiddenparmaGiuseppe Lavagettooperations/puppetproduction+11 -0
hiddenparma: add default ratelimits fileGiuseppe Lavagettooperations/puppetproduction+15 -1
haproxy: use ipblocks map created by hiddenparmaGiuseppe Lavagettooperations/puppetproduction+8 -47
fetch_external_clouds_vendors_nets: commit changes to provenance mapGiuseppe Lavagettooperations/puppetproduction+23 -0
haproxy: get ipblock map directly from HPGiuseppe Lavagettooperations/puppetproduction+18 -0
Customize query in gerrit
Related Changes in GitLab:
TitleReferenceAuthorSource BranchDest Branch
Override ipblock scope during aggregationrepos/sre/hiddenparma!169swfrenchwork/swfrench/ipblock-aggregation-scopesmain
Aggregate ipblock CIDRs using a binary trierepos/sre/hiddenparma!168swfrenchwork/swfrench/ipblock-aggregationmain
Generate known-client ratelimit stanzas for varnish in HPrepos/sre/hiddenparma!166oblivianknown_client_ratelimitsmain
New type of derived data: haproxy_provenance_maprepos/sre/hiddenparma!149oblivianhaproxy_ipblock_mapmain
Customize query in GitLab

Related Objects
Search...

Event Timeline

Joe created this task.Apr 3 2026, 1:16 PM
CodeReviewBot added a comment.Jun 6 2026, 5:07 AM

oblivian updated https://gitlab.wikimedia.org/repos/sre/hiddenparma/-/merge_requests/149

New type of derived data: haproxy_provenance_map

gerritbot added a comment.Jun 10 2026, 5:56 AM

Change #1299939 had a related patch set uploaded (by Giuseppe Lavagetto; author: Giuseppe Lavagetto):

[operations/puppet@production] haproxy: get ipblock map directly from HP

https://gerrit.wikimedia.org/r/1299939

gerritbot added a project: Patch-For-Review.Jun 10 2026, 5:56 AM

Change #1299940 had a related patch set uploaded (by Giuseppe Lavagetto; author: Giuseppe Lavagetto):

[operations/puppet@production] haproxy: use ipblocks map created by hiddenparma

https://gerrit.wikimedia.org/r/1299940

Joe triaged this task as Medium priority.Jun 12 2026, 4:19 AM
CodeReviewBot added a comment.Jun 12 2026, 2:23 PM

oblivian opened https://gitlab.wikimedia.org/repos/sre/hiddenparma/-/merge_requests/166

Generate known-client ratelimit stanzas for varnish in HP

gerritbot added a comment.Mon, Jun 15, 6:02 AM

Change #1301883 had a related patch set uploaded (by Giuseppe Lavagetto; author: Giuseppe Lavagetto):

[operations/puppet@production] fetch_external_clouds_vendors_nets: commit changes to provenance map

https://gerrit.wikimedia.org/r/1301883

CodeReviewBot added a comment.Mon, Jun 15, 6:33 AM

oblivian merged https://gitlab.wikimedia.org/repos/sre/hiddenparma/-/merge_requests/149

New type of derived data: haproxy_provenance_map

CodeReviewBot added a comment.Mon, Jun 15, 11:37 PM

swfrench opened https://gitlab.wikimedia.org/repos/sre/hiddenparma/-/merge_requests/168

Aggregate ipblock CIDRs using a binary trie

CodeReviewBot added a comment.Tue, Jun 16, 3:20 AM

swfrench opened https://gitlab.wikimedia.org/repos/sre/hiddenparma/-/merge_requests/169

Override ipblock scope during aggregation

CodeReviewBot added a comment.Wed, Jun 17, 12:16 AM

swfrench merged https://gitlab.wikimedia.org/repos/sre/hiddenparma/-/merge_requests/168

Aggregate ipblock CIDRs using a binary trie

CodeReviewBot added a comment.Wed, Jun 17, 12:24 AM

swfrench merged https://gitlab.wikimedia.org/repos/sre/hiddenparma/-/merge_requests/169

Override ipblock scope during aggregation

gerritbot added a comment.Wed, Jun 17, 7:34 AM

Change #1299939 merged by Giuseppe Lavagetto:

[operations/puppet@production] haproxy: get ipblock map directly from HP

https://gerrit.wikimedia.org/r/1299939

gerritbot added a comment.Wed, Jun 17, 8:44 AM

Change #1301883 merged by Giuseppe Lavagetto:

[operations/puppet@production] fetch_external_clouds_vendors_nets: commit changes to provenance map

https://gerrit.wikimedia.org/r/1301883

gerritbot added a comment.Wed, Jun 17, 1:44 PM

Change #1299940 merged by Giuseppe Lavagetto:

[operations/puppet@production] haproxy: use ipblocks map created by hiddenparma

https://gerrit.wikimedia.org/r/1299940

CodeReviewBot added a comment.Thu, Jun 25, 12:27 PM

oblivian merged https://gitlab.wikimedia.org/repos/sre/hiddenparma/-/merge_requests/166

Generate known-client ratelimit stanzas for varnish in HP

gerritbot added a comment.Tue, Jun 30, 8:05 AM

Change #1306501 had a related patch set uploaded (by Giuseppe Lavagetto; author: Giuseppe Lavagetto):

[operations/puppet@production] hiddenparma: add default ratelimits file

https://gerrit.wikimedia.org/r/1306501

gerritbot added a comment.Tue, Jun 30, 8:05 AM

Change #1306502 had a related patch set uploaded (by Giuseppe Lavagetto; author: Giuseppe Lavagetto):

[operations/puppet@production] cache::varnish: add rate-limit file generated from hiddenparma

https://gerrit.wikimedia.org/r/1306502

gerritbot added a comment.Tue, Jun 30, 8:05 AM

Change #1306503 had a related patch set uploaded (by Giuseppe Lavagetto; author: Giuseppe Lavagetto):

[operations/puppet@production] cache::varnish: switch known client rate limits to hp-generated data

https://gerrit.wikimedia.org/r/1306503

gerritbot added a comment.Wed, Jul 1, 6:10 AM

Change #1306501 merged by Giuseppe Lavagetto:

[operations/puppet@production] hiddenparma: add default ratelimits file

https://gerrit.wikimedia.org/r/1306501

gerritbot added a comment.Wed, Jul 1, 9:36 AM

Change #1306502 merged by Giuseppe Lavagetto:

[operations/puppet@production] cache::varnish: add rate-limit file generated from hiddenparma

https://gerrit.wikimedia.org/r/1306502

gerritbot added a comment.Wed, Jul 1, 1:51 PM

Change #1306503 merged by Giuseppe Lavagetto:

[operations/puppet@production] cache::varnish: switch known client rate limits to hp-generated data

https://gerrit.wikimedia.org/r/1306503

Joe closed this task as Resolved.Fri, Jul 3, 8:45 AM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits