Page MenuHomePhabricator

Update designate sink plugins to work with caracal
Closed, ResolvedPublic

Description

Designate release notes say:

Removed the following unused central rpc calls. This should not impact normal installations, but if these are used in any custom written backends or plugins that you are using, you will need to update your code before upgrading.

    find_records

    find_recordset

    update_zone_import

We at least use find_records and possibly also find_recordsets.

Event Timeline

Oh, good news, I already did this all but one (possibly tricky) case.

Change #1059409 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] wmf_sink: rip out the proxy-cleanup code

https://gerrit.wikimedia.org/r/1059409

The remaining case is in wmf_sink, the proxy cleanup code. That code relies on being able to look up the dns records (and, by extension, the IPs) associated with a deleted VM. That is no longer possible.

As an alternative I'm adding an endpoint to the dynamic-proxy api which will purges all dangling proxies from a project. It will also delete any proxies that point outside of a project -- as best I can tell we don't do that anyway, nor should we.

There are a whole lot of dangling proxies. Here's the report on what will ultimately get cleaned up:

Would delete equitylandscape.wmcloud.org, http://172.16.0.27:80
   ---------------- which points to a real VM: canary1057-2.cloudvirt-canary
Would delete beta-prometheus.wmflabs.org, http://172.16.2.236:80
   ---------------- which points to a real VM: mdwiki.mdwikioffline
Would delete citoid-beta.wmflabs.org, http://172.16.4.222:1970
Would delete parsoid-beta.wmflabs.org, http://172.16.4.125:80
Would delete parsoid-external-ci-access.wmcloud.org, http://172.16.4.125:80
Would delete prometheus-beta.wmcloud.org, http://172.16.2.236:80
   ---------------- which points to a real VM: mdwiki.mdwikioffline
Would delete pushnotifications.wmflabs.org, http://172.16.1.215:8900
Would delete shellbox-beta.wmcloud.org, http://172.16.4.150:8080
Would delete w-beta.wmflabs.org, http://172.16.3.203:80
Would delete jenkins-releases.wmcloud.org, http://172.16.0.235:8080
Would delete dashboard-replica-endpoint.wmcloud.org, http://172.16.5.193:80
Would delete outreachdashboard-web.wmcloud.org, http://172.16.2.18:80
Would delete peony.wmflabs.org, http://172.16.2.18:80
Would delete huggle-wl.wmflabs.org, http://172.16.2.30:80
Would delete staticbz.wmcloud.org, http://172.16.5.122:8080
Would delete opusmt.wmflabs.org, http://172.16.0.115:8888
Would delete tesseract.wmcloud.org, http://172.16.0.115:5678
Would delete tts.wmcloud.org, http://172.16.0.115:5002
Would delete wq.wmcloud.org, http://172.16.0.115:8000
Would delete corenlp-test.wmflabs.org, http://172.16.5.156:9000
Would delete mixnmatch-qs.wmflabs.org, http://172.16.1.251:9191
Would delete mixnmatch-query.wmflabs.org, http://172.16.1.251:8282
Would delete mixnmatch.wmflabs.org, http://172.16.1.251:8181
Would delete dispatch-demo.wmcloud.org, http://172.16.3.202:8000
Would delete dispatch-keith.wmcloud.org, http://172.16.0.171:8000
   ---------------- which points to a real VM: superset-127-jxhvhh7bzlrl-node-0.superset
Would delete pdev-puppetboard.wmcloud.org, http://172.16.6.60:80
Would delete quarry-dev.wmflabs.org, http://172.16.5.58:80
Would delete quarry-kube.wmcloud.org, http://172.16.5.165:31597
Would delete beryllium.wmflabs.org, http://172.16.5.77:80
   ---------------- which points to a real VM: k3s.catalyst
Would delete gitlab.wmflabs.org, http://172.16.3.180:80
Would delete neon.wmflabs.org, http://172.16.3.180:80
Would delete vanadium.wmflabs.org, http://172.16.1.54:3000
Would delete armadillo.wmcloud.org, http://172.16.5.135:8080
Would delete pixel-old.wmcloud.org, http://172.16.3.111:80
Would delete pixel-test.wmcloud.org, http://172.16.3.111:80
Would delete similar-image.wmcloud.org, http://172.16.5.249:80
Would delete test-horizon.wmcloud.org, http://172.16.2.55:80
Would delete cirrustest-cirrus-integ.wmflabs.org, http://172.16.0.133:8080
   ---------------- which points to a real VM: tools-k8s-worker-nfs-4.tools
Would delete commons-cirrus-integ.wmflabs.org, http://172.16.0.133:8080
   ---------------- which points to a real VM: tools-k8s-worker-nfs-4.tools
Would delete commons-defaults-relforge.wmflabs.org, http://172.16.2.61:8080
Would delete commons-img-qual-relforge.wmflabs.org, http://172.16.2.61:8080
Would delete discernatron.wmflabs.org, http://172.16.2.80:8080
Would delete ru-cirrus-integ.wmflabs.org, http://172.16.0.133:8080
   ---------------- which points to a real VM: tools-k8s-worker-nfs-4.tools
Would delete idp-dev.wmcloud.org, http://172.16.5.197:8080
Would delete idp-test-login.wmcloud.org, http://172.16.5.197:80
Would delete sso-debmon.wmcloud.org, http://172.16.2.174:80
Would delete sso-django-login.wmcloud.org, http://172.16.5.197:80
Would delete sso-gitlab.wmcloud.org, http://172.16.5.159:80
   ---------------- which points to a real VM: gitlab-runner-addshore-1017.mwcli
Would delete sso-idp.wmcloud.org, http://172.16.2.183:8080
   ---------------- which points to a real VM: tools-k8s-etcd-24.tools
Would delete andrewfuturetest.wmcloud.org, http://172.16.3.16:80
Would delete andrewlegacytest1.wmflabs.org, http://172.16.3.16:80
Would delete deletemetest.wmcloud.org, http://172.16.4.140:80
   ---------------- which points to a real VM: canary1045-3.cloudvirt-canary
Would delete flask-test.wmcloud.org, http://172.16.5.60:80
Would delete taavi-intentionally-broken.wmcloud.org, http://172.16.6.255:1234
Would delete taavi-loki-grafana.wmcloud.org, http://172.16.2.154:80
Would delete testlabshorizon.wmcloud.org, http://172.16.0.136:80
Would delete wmcloudtest.wmcloud.org, http://172.16.3.16:80
Would delete querysort.wmcloud.org, http://172.16.6.150:80
Would delete videowiki-test.wmcloud.org, http://172.16.6.78:80
   ---------------- which points to a real VM: peony-database.globaleducation
Would delete patchdemo2.wmcloud.org, http://172.16.6.170:80
   ---------------- which points to a real VM: deployment-jobrunner05.deployment-prep
Would delete visualeditor-prototype.wmflabs.org, http://172.16.0.13:8080
Would delete visualeditor-realtime.wmflabs.org, http://172.16.0.130:8081
   ---------------- which points to a real VM: video-redis-bookworm.video
Would delete visualeditor-test.wmflabs.org, http://172.16.0.130:8080
   ---------------- which points to a real VM: video-redis-bookworm.video
Would delete opennext-federated-properties.wmflabs.org, http://172.16.2.43:80
Would delete opennext-query.wmflabs.org, http://172.16.2.43:8834
Would delete queripulator.wmcloud.org, http://172.16.2.33:8080
Would delete wbinstalltest.wmcloud.org, http://172.16.1.79:80
   ---------------- which points to a real VM: deployment-maps-master02.deployment-prep
Would delete wikibase-product-testing-fedprops-previous.wmcloud.org, http://172.16.6.128:8480
Would delete wikibase-product-testing-fedprops.wmcloud.org, http://172.16.6.128:8580
Would delete wikibase-product-testing-previous.wmcloud.org, http://172.16.6.128:8280
Would delete wikibase-product-testing.wmcloud.org, http://172.16.6.128:8380
Would delete wikibase-qs-testing-fedprops-previous.wmcloud.org, http://172.16.6.128:8482
Would delete wikibase-qs-testing-fedprops.wmcloud.org, http://172.16.6.128:8582
Would delete wikibase-qs-testing-previous.wmcloud.org, http://172.16.6.128:8282
Would delete wikibase-qs-testing.wmcloud.org, http://172.16.6.128:8382
Would delete wikibase-query-testing-fedprops-previous.wmcloud.org, http://172.16.6.128:8481
Would delete wikibase-query-testing-fedprops.wmcloud.org, http://172.16.6.128:8581
Would delete wikibase-query-testing-previous.wmcloud.org, http://172.16.6.128:8281
Would delete wikibase-query-testing.wmcloud.org, http://172.16.6.128:8381
Would delete wikibase-reconcile-testing.wmcloud.org, http://172.16.5.146:80
Would delete wikidata-federated-properties.wmflabs.org, http://172.16.2.212:80
Would delete wdhqs.wmflabs.org, http://172.16.1.123:80
Would delete denelezh.wmcloud.org, http://172.16.0.137:80
Would delete humaniki-staging.wmcloud.org, http://172.16.0.215:80
   ---------------- which points to a real VM: ia-upload-test2.wikisource
Would delete whgi.wmflabs.org, http://172.16.0.137:80
Would delete wikispeech-symbolset.wmcloud.org, http://172.16.1.24:8771
Would delete wikispeech-tts.wmflabs.org, http://172.16.1.24:10000
Would delete covid-data.wmflabs.org, http://172.16.0.139:80
Would delete embeddings-experiments.wmcloud.org, http://172.16.5.101:80
   ---------------- which points to a real VM: owmf-test.wikispore
Would delete secrec.wmflabs.org, http://172.16.0.105:80
Would delete wikidata-wikipedia.wmflabs.org, http://172.16.2.49:80

Change #1059958 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] dynamic proxy: Add an endpoint for scrubbing out nonexistent backends

https://gerrit.wikimedia.org/r/1059958

Change #1059959 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] wmf_sink: replace targetted proxy cleanup with project-wide cleanup

https://gerrit.wikimedia.org/r/1059959

Change #1059409 abandoned by Andrew Bogott:

[operations/puppet@production] wmf_sink: rip out the proxy-cleanup code

Reason:

dropping in favor of https://gerrit.wikimedia.org/r/c/operations/puppet/+/1059959

https://gerrit.wikimedia.org/r/1059409

Change #1060172 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] wmfsink: hook delete.end rather than delete.start

https://gerrit.wikimedia.org/r/1060172

Change #1060172 merged by Andrew Bogott:

[operations/puppet@production] wmfsink: hook delete.end rather than delete.start

https://gerrit.wikimedia.org/r/1060172

Change #1059958 merged by Andrew Bogott:

[operations/puppet@production] dynamic proxy: Add an endpoint for scrubbing out nonexistent backends

https://gerrit.wikimedia.org/r/1059958

Change #1059959 merged by Andrew Bogott:

[operations/puppet@production] wmf_sink: replace targeted proxy cleanup with project-wide cleanup

https://gerrit.wikimedia.org/r/1059959