Page MenuHomePhabricator

Merge cache_maps into cache_upload functionally
Closed, ResolvedPublic

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

Change 351663 had a related patch set uploaded (by BBlack; owner: BBlack):
[operations/puppet@production] maps->upload functional cluster-level changes

https://gerrit.wikimedia.org/r/351663

Change 352834 had a related patch set uploaded (by BBlack; owner: BBlack):
[operations/puppet@production] maps->upload: move IP, delete maps-specific things

https://gerrit.wikimedia.org/r/352834

@elukey - I think the only real analytics fallout here is that the data that is currently feeding to you as webrequest_maps will become data that's mixed into the existing feed of webrequest_upload. They'll still be differentiated on the request hostname (upload.wikimedia.org vs maps.wikimedia.org). At the time of deploy for the final transition commit, the data would move over smoothly from one to the other over a period of several minutes. Is this something that requires some special accommodation on analytics end first? Timeline?

I am going to ask to my team and report back asap!

Change 353054 had a related patch set uploaded (by BBlack; owner: BBlack):
[operations/puppet@production] maps->upload: move LVS IPs

https://gerrit.wikimedia.org/r/353054

Change 351663 merged by BBlack:
[operations/puppet@production] maps->upload functional cluster-level changes

https://gerrit.wikimedia.org/r/351663

Change 353069 had a related patch set uploaded (by BBlack; owner: BBlack):
[operations/puppet@production] maps->upload: fix kartotherian be_opts

https://gerrit.wikimedia.org/r/353069

Change 353069 merged by BBlack:
[operations/puppet@production] maps->upload: fix kartotherian be_opts

https://gerrit.wikimedia.org/r/353069

Change 353073 had a related patch set uploaded (by BBlack; owner: BBlack):
[operations/puppet@production] maps->upload: keep the upload sec-related headers upload-only

https://gerrit.wikimedia.org/r/353073

Change 353073 merged by BBlack:
[operations/puppet@production] maps->upload: keep the upload sec-related headers upload-only

https://gerrit.wikimedia.org/r/353073

I don't think that this should cause any problems on our side. I'm not aware of any maps specific jobs we run. @JAllemandou let's remember to remove the maps load/refine jobs from oozie after this is done.

Mentioned in SAL (#wikimedia-operations) [2017-05-31T15:31:46Z] <ema> merge cache_maps into cache_upload: move LVS IPs T164608

Change 353054 merged by Ema:
[operations/puppet@production] maps->upload: move LVS IPs

https://gerrit.wikimedia.org/r/353054

Mentioned in SAL (#wikimedia-operations) [2017-05-31T16:53:10Z] <ema> merge cache_maps into cache_upload: finished moving LVS IPs T164608

Change 356570 had a related patch set uploaded (by Volans; owner: Volans):
[operations/puppet@production] Maps cache: fix parameters stripped away

https://gerrit.wikimedia.org/r/356570

Change 356570 merged by Ema:
[operations/puppet@production] Maps cache: fix parameters stripped away

https://gerrit.wikimedia.org/r/356570

Change 356583 had a related patch set uploaded (by Ema; owner: Ema):
[operations/puppet@production] varnish: add explicit guards around upload-specific VCL

https://gerrit.wikimedia.org/r/356583

Change 356583 merged by BBlack:
[operations/puppet@production] varnish: add explicit guards around upload-specific VCL

https://gerrit.wikimedia.org/r/356583

Change 356819 had a related patch set uploaded (by BBlack; owner: BBlack):
[operations/dns@master] maps: coalesce DNS to upload IPs

https://gerrit.wikimedia.org/r/356819

Change 356819 merged by BBlack:
[operations/dns@master] maps: coalesce DNS to upload IPs

https://gerrit.wikimedia.org/r/356819

Change 357632 had a related patch set uploaded (by BBlack; owner: BBlack):
[operations/dns@master] remove maps geoip definition

https://gerrit.wikimedia.org/r/357632

Change 357634 had a related patch set uploaded (by BBlack; owner: BBlack):
[operations/dns@master] remove old maps reverse dns

https://gerrit.wikimedia.org/r/357634

Change 357635 had a related patch set uploaded (by BBlack; owner: BBlack):
[operations/puppet@production] lvs_service_ips: remove old maps IPs

https://gerrit.wikimedia.org/r/357635

Change 352834 merged by BBlack:
[operations/puppet@production] maps->upload: delete maps-specific things

https://gerrit.wikimedia.org/r/352834

Change 357635 merged by BBlack:
[operations/puppet@production] lvs_service_ips: remove old maps IPs

https://gerrit.wikimedia.org/r/357635

Change 357632 merged by BBlack:
[operations/dns@master] remove maps geoip definition

https://gerrit.wikimedia.org/r/357632

Change 357634 merged by BBlack:
[operations/dns@master] remove old maps reverse dns

https://gerrit.wikimedia.org/r/357634

Change 357725 had a related patch set uploaded (by BBlack; owner: BBlack):
[operations/puppet@production] cache_maps: remove ipsec defs on kafka nodes

https://gerrit.wikimedia.org/r/357725

Change 357725 merged by BBlack:
[operations/puppet@production] cache_maps: remove ipsec defs on kafka nodes

https://gerrit.wikimedia.org/r/357725

Change 357729 had a related patch set uploaded (by BBlack; owner: BBlack):
[operations/puppet@production] cache_maps: remove other misc refs

https://gerrit.wikimedia.org/r/357729

Change 357729 merged by BBlack:
[operations/puppet@production] cache_maps: remove other misc refs

https://gerrit.wikimedia.org/r/357729

Script wmf_auto_reimage was launched by bblack on neodymium.eqiad.wmnet for hosts:

['cp3004.esams.wmnet', 'cp3005.esams.wmnet', 'cp3006.esams.wmnet', 'cp4011.ulsfo.wmnet', 'cp4012.ulsfo.wmnet', 'cp4019.ulsfo.wmnet', 'cp4020.ulsfo.wmnet', 'cp2003.codfw.wmnet', 'cp2009.codfw.wmnet', 'cp2015.codfw.wmnet', 'cp2021.codfw.wmnet', 'cp1046.eqiad.wmnet', 'cp1047.eqiad.wmnet', 'cp1059.eqiad.wmnet', 'cp1060.eqiad.wmnet']

The log can be found in /var/log/wmf-auto-reimage/201706072224_bblack_11757.log.

Change 357734 had a related patch set uploaded (by BBlack; owner: BBlack):
[operations/puppet@production] site.pp: commentary for ex-cache_maps hosts

https://gerrit.wikimedia.org/r/357734

Change 357734 merged by BBlack:
[operations/puppet@production] site.pp: commentary for ex-cache_maps hosts

https://gerrit.wikimedia.org/r/357734

Script wmf_auto_reimage was launched by bblack on neodymium.eqiad.wmnet for hosts:

['cp3004.esams.wmnet', 'cp3005.esams.wmnet', 'cp4011.ulsfo.wmnet', 'cp4012.ulsfo.wmnet', 'cp4019.ulsfo.wmnet', 'cp4020.ulsfo.wmnet', 'cp2009.codfw.wmnet', 'cp2021.codfw.wmnet', 'cp1046.eqiad.wmnet', 'cp1059.eqiad.wmnet']

The log can be found in /var/log/wmf-auto-reimage/201706072303_bblack_26511.log.

Script wmf_auto_reimage was launched by bblack on neodymium.eqiad.wmnet for hosts:

['cp3006.esams.wmnet', 'cp1046.eqiad.wmnet', 'cp1047.eqiad.wmnet', 'cp2009.codfw.wmnet']

The log can be found in /var/log/wmf-auto-reimage/201706080254_bblack_7845.log.

BBlack claimed this task.

Change 460274 had a related patch set uploaded (by Ema; owner: Ema):
[operations/dns@master] Remove maps-lb definitions

https://gerrit.wikimedia.org/r/460274

Change 460274 merged by Ema:
[operations/dns@master] Remove maps-lb definitions

https://gerrit.wikimedia.org/r/460274