Page MenuHomePhabricator

Merge cache_maps into cache_upload functionally
Closed, ResolvedPublic

Details

Related Gerrit Patches:
operations/dns : masterRemove maps-lb definitions
operations/puppet : productionsite.pp: commentary for ex-cache_maps hosts
operations/puppet : productioncache_maps: remove other misc refs
operations/puppet : productioncache_maps: remove ipsec defs on kafka nodes
operations/dns : masterremove old maps reverse dns
operations/dns : masterremove maps geoip definition
operations/puppet : productionlvs_service_ips: remove old maps IPs
operations/puppet : productionmaps->upload: delete maps-specific things
operations/dns : mastermaps: coalesce DNS to upload IPs
operations/puppet : productionvarnish: add explicit guards around upload-specific VCL
operations/puppet : productionMaps cache: fix parameters stripped away
operations/puppet : productionmaps->upload: move LVS IPs
operations/puppet : productionmaps->upload: keep the upload sec-related headers upload-only
operations/puppet : productionmaps->upload: fix kartotherian be_opts
operations/puppet : productionmaps->upload functional cluster-level changes

Event Timeline

BBlack created this task.May 5 2017, 5:56 PM
Restricted Application added a project: Operations. · View Herald TranscriptMay 5 2017, 5:56 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
ema moved this task from Triage to Caching on the Traffic board.May 9 2017, 8:36 AM

Change 351663 had a related patch set uploaded (by BBlack; owner: BBlack):
[operations/puppet@production] maps->upload functional cluster-level changes

https://gerrit.wikimedia.org/r/351663

Change 352834 had a related patch set uploaded (by BBlack; owner: BBlack):
[operations/puppet@production] maps->upload: move IP, delete maps-specific things

https://gerrit.wikimedia.org/r/352834

BBlack added a subscriber: elukey.May 9 2017, 2:48 PM

@elukey - I think the only real analytics fallout here is that the data that is currently feeding to you as webrequest_maps will become data that's mixed into the existing feed of webrequest_upload. They'll still be differentiated on the request hostname (upload.wikimedia.org vs maps.wikimedia.org). At the time of deploy for the final transition commit, the data would move over smoothly from one to the other over a period of several minutes. Is this something that requires some special accommodation on analytics end first? Timeline?

elukey added a comment.May 9 2017, 2:49 PM

I am going to ask to my team and report back asap!

Change 353054 had a related patch set uploaded (by BBlack; owner: BBlack):
[operations/puppet@production] maps->upload: move LVS IPs

https://gerrit.wikimedia.org/r/353054

Change 351663 merged by BBlack:
[operations/puppet@production] maps->upload functional cluster-level changes

https://gerrit.wikimedia.org/r/351663

Change 353069 had a related patch set uploaded (by BBlack; owner: BBlack):
[operations/puppet@production] maps->upload: fix kartotherian be_opts

https://gerrit.wikimedia.org/r/353069

Change 353069 merged by BBlack:
[operations/puppet@production] maps->upload: fix kartotherian be_opts

https://gerrit.wikimedia.org/r/353069

Change 353073 had a related patch set uploaded (by BBlack; owner: BBlack):
[operations/puppet@production] maps->upload: keep the upload sec-related headers upload-only

https://gerrit.wikimedia.org/r/353073

Change 353073 merged by BBlack:
[operations/puppet@production] maps->upload: keep the upload sec-related headers upload-only

https://gerrit.wikimedia.org/r/353073

Ottomata added a subscriber: Ottomata.EditedMay 17 2017, 8:06 AM

I don't think that this should cause any problems on our side. I'm not aware of any maps specific jobs we run. @JAllemandou let's remember to remove the maps load/refine jobs from oozie after this is done.

Mentioned in SAL (#wikimedia-operations) [2017-05-31T15:31:46Z] <ema> merge cache_maps into cache_upload: move LVS IPs T164608

Change 353054 merged by Ema:
[operations/puppet@production] maps->upload: move LVS IPs

https://gerrit.wikimedia.org/r/353054

Mentioned in SAL (#wikimedia-operations) [2017-05-31T16:53:10Z] <ema> merge cache_maps into cache_upload: finished moving LVS IPs T164608

Change 356570 had a related patch set uploaded (by Volans; owner: Volans):
[operations/puppet@production] Maps cache: fix parameters stripped away

https://gerrit.wikimedia.org/r/356570

Change 356570 merged by Ema:
[operations/puppet@production] Maps cache: fix parameters stripped away

https://gerrit.wikimedia.org/r/356570

Change 356583 had a related patch set uploaded (by Ema; owner: Ema):
[operations/puppet@production] varnish: add explicit guards around upload-specific VCL

https://gerrit.wikimedia.org/r/356583

Change 356583 merged by BBlack:
[operations/puppet@production] varnish: add explicit guards around upload-specific VCL

https://gerrit.wikimedia.org/r/356583

Change 356819 had a related patch set uploaded (by BBlack; owner: BBlack):
[operations/dns@master] maps: coalesce DNS to upload IPs

https://gerrit.wikimedia.org/r/356819

Change 356819 merged by BBlack:
[operations/dns@master] maps: coalesce DNS to upload IPs

https://gerrit.wikimedia.org/r/356819

Change 357632 had a related patch set uploaded (by BBlack; owner: BBlack):
[operations/dns@master] remove maps geoip definition

https://gerrit.wikimedia.org/r/357632

Change 357634 had a related patch set uploaded (by BBlack; owner: BBlack):
[operations/dns@master] remove old maps reverse dns

https://gerrit.wikimedia.org/r/357634

Change 357635 had a related patch set uploaded (by BBlack; owner: BBlack):
[operations/puppet@production] lvs_service_ips: remove old maps IPs

https://gerrit.wikimedia.org/r/357635

Change 352834 merged by BBlack:
[operations/puppet@production] maps->upload: delete maps-specific things

https://gerrit.wikimedia.org/r/352834

Change 357635 merged by BBlack:
[operations/puppet@production] lvs_service_ips: remove old maps IPs

https://gerrit.wikimedia.org/r/357635

Change 357632 merged by BBlack:
[operations/dns@master] remove maps geoip definition

https://gerrit.wikimedia.org/r/357632

Change 357634 merged by BBlack:
[operations/dns@master] remove old maps reverse dns

https://gerrit.wikimedia.org/r/357634

Change 357725 had a related patch set uploaded (by BBlack; owner: BBlack):
[operations/puppet@production] cache_maps: remove ipsec defs on kafka nodes

https://gerrit.wikimedia.org/r/357725

Change 357725 merged by BBlack:
[operations/puppet@production] cache_maps: remove ipsec defs on kafka nodes

https://gerrit.wikimedia.org/r/357725

Change 357729 had a related patch set uploaded (by BBlack; owner: BBlack):
[operations/puppet@production] cache_maps: remove other misc refs

https://gerrit.wikimedia.org/r/357729

Change 357729 merged by BBlack:
[operations/puppet@production] cache_maps: remove other misc refs

https://gerrit.wikimedia.org/r/357729

Script wmf_auto_reimage was launched by bblack on neodymium.eqiad.wmnet for hosts:

['cp3004.esams.wmnet', 'cp3005.esams.wmnet', 'cp3006.esams.wmnet', 'cp4011.ulsfo.wmnet', 'cp4012.ulsfo.wmnet', 'cp4019.ulsfo.wmnet', 'cp4020.ulsfo.wmnet', 'cp2003.codfw.wmnet', 'cp2009.codfw.wmnet', 'cp2015.codfw.wmnet', 'cp2021.codfw.wmnet', 'cp1046.eqiad.wmnet', 'cp1047.eqiad.wmnet', 'cp1059.eqiad.wmnet', 'cp1060.eqiad.wmnet']

The log can be found in /var/log/wmf-auto-reimage/201706072224_bblack_11757.log.

Change 357734 had a related patch set uploaded (by BBlack; owner: BBlack):
[operations/puppet@production] site.pp: commentary for ex-cache_maps hosts

https://gerrit.wikimedia.org/r/357734

Change 357734 merged by BBlack:
[operations/puppet@production] site.pp: commentary for ex-cache_maps hosts

https://gerrit.wikimedia.org/r/357734

Script wmf_auto_reimage was launched by bblack on neodymium.eqiad.wmnet for hosts:

['cp3004.esams.wmnet', 'cp3005.esams.wmnet', 'cp4011.ulsfo.wmnet', 'cp4012.ulsfo.wmnet', 'cp4019.ulsfo.wmnet', 'cp4020.ulsfo.wmnet', 'cp2009.codfw.wmnet', 'cp2021.codfw.wmnet', 'cp1046.eqiad.wmnet', 'cp1059.eqiad.wmnet']

The log can be found in /var/log/wmf-auto-reimage/201706072303_bblack_26511.log.

Script wmf_auto_reimage was launched by bblack on neodymium.eqiad.wmnet for hosts:

['cp3006.esams.wmnet', 'cp1046.eqiad.wmnet', 'cp1047.eqiad.wmnet', 'cp2009.codfw.wmnet']

The log can be found in /var/log/wmf-auto-reimage/201706080254_bblack_7845.log.

BBlack closed this task as Resolved.Jun 8 2017, 3:51 AM
BBlack claimed this task.

Change 460274 had a related patch set uploaded (by Ema; owner: Ema):
[operations/dns@master] Remove maps-lb definitions

https://gerrit.wikimedia.org/r/460274

Change 460274 merged by Ema:
[operations/dns@master] Remove maps-lb definitions

https://gerrit.wikimedia.org/r/460274