Page MenuHomePhabricator

Remove old scap repositories from deploy1002
Open, MediumPublic

Description

As described in T307349#7895775, it would be nice to clean up old repositories on deploy1002 to avoid inconsistencies when comparing deploy1002 with deploy2002.

Event Timeline

The number of repositories on deploy1002 and deploy2002 under /srv/deployment is the same (as intended by the rsync, it uses --delete). The total size also looks the same.

This ticket sounds like this wouldn't be the case though?

Top ten oldest repos by modifiation time, oldest first:

Oct  9  2013 elasticsearch
Dec  9  2013 scholarships
Apr 18  2014 ocg
May 30  2014 rcstream
Sep  9  2014 servermon
Oct 17  2014 iegreview
Dec  9  2014 statsv
Feb 12  2015 cxserver
Mar  9  2015 zotero

(same on both servers)

list of repositories mentioned in hieradata/role/common/deployment_server/kubernetes.yaml

(some have a repository but also a "scap_repository" value?)

repository: analytics/statsv
repository: data-engineering/airflow-dags
repository: data-engineering/airflow-dags
repository: data-engineering/airflow-dags
repository: eventlogging
repository: iegreview
repository: integration/zuul/deploy
repository: labs/striker/deploy
repository: maps/kartotherian/deploy
repository: maps/tilerator/deploy
repository: mediawiki/services/eventstreams/deploy
repository: mediawiki/services/ores/deploy
repository: mediawiki/services/parsoid/deploy
repository: mediawiki/services/restbase/deploy
repository: openstack/horizon/deploy
repository: operations/docker-images/docker-pkg/deploy
repository: operations/dumps
repository: operations/software/cassandra-twcs
repository: operations/software/debmonitor/deploy
repository: operations/software/gerrit
repository: operations/software/gerrit/tools/gervert/deploy
repository: operations/software/homer/deploy
repository: operations/software/librenms
repository: operations/software/logstash-logback-encoder
repository: operations/software/logstash/plugins
repository: operations/software/netbox-deploy
repository: search/MjoLniR/deploy
repository: wikidata/query/deploy
repository: wikimedia/discovery/analytics
scap_repository: analytics/refinery/scap
scap_repository: data-engineering/airflow-dags-scap-analytics
scap_repository: data-engineering/airflow-dags-scap-analytics_test
scap_repository: data-engineering/airflow-dags-scap-research
scap_repository: eventlogging/scap/analytics
scap_repository: operations/dumps/scap

So does T307349#7895775 say all repos on (both) deployment servers should be deleted if they do not appear in the list above?

list of repos that exist on deployment servers but do not appear in the kubernetes.yaml. (just using the string that is the first level of the name but if this does not appear at all in the yaml then it's a candidate).

3d2png
apache2modsec
changeprop
citoid
cp-jobqueue
cpjobqueue
cxserver
design
dropwizard
elasticsearch
electron-render
fluoride
graphoid
httpbb
httpbb-tests
imagecatalog
jobrunner
jobrunner.old
mathoid
mediawiki-staging
mobileapps
modsec
ocg
performance
phabricator
prometheus
proton
puppetboard
rcstream
recommendation-api
releng
relforge
scholarships
sentry
servermon
trending-edits
wdqs
zotero

@Dzahn That doesn't seem right- mediawiki-staging is the current main method of deploying mediawiki, and httpbb-tests seems in active usage (although I don't know how it is deployed): https://icinga.wikimedia.org/cgi-bin/icinga/status.cgi?search_string=httpbb

Both appear on puppet:

https://puppetboard.wikimedia.org/catalog/deploy1002.eqiad.wmnet

My suggestion is to start with those resources not appearing on puppet first.

Those that are old AND on puppet should be easier to cleanup with a puppet ensure => absent.

This is a list of resources configured on puppet, but I am not sure if the list is exhaustive:

File[/srv/deployment/scap] from /etc/puppet/modules/git/manifests/clone.pp:184
File[/srv/deployment/httpbb-tests] from /etc/puppet/modules/httpbb/manifests/init.pp:13
File[/srv/deployment/imagecatalog] from /etc/puppet/modules/imagecatalog/manifests/init.pp:34
File[/srv/deployment/mediawiki-staging] from /etc/puppet/modules/profile/manifests/mediawiki/deployment/server.pp:122

@jcrespo You are correct. In that case I still don't understand what this ticket is really asking for, first I thought it was about both deployment servers not being in sync (T309162#7958748), then I thought it was about the kubernetes.yaml not being in sync with what is on the deployment servers. It doesn't seem to be either of those though.