Page MenuHomePhabricator
Create Task
Maniphest T267578

Puppet disabled in beta cluster varnish deployment-cache-text06
Closed, DeclinedPublic
Actions

Description

Beta cluster was down from ??? to Mon, 09 Nov 2020 16:21:17 +0000

I believe this was related to T267439: MediaWiki beta varnish is down., but I'm not entirely certain.

Current status:

  • Puppet disabled on beta's varnish node since it seems to re-break beta, currently
  • Pinned varnish debian versions:
    • varnish=5.1.3-1wm15
    • varnish-modules=0.12.1-1+wmf2
    • libvmod-netmapper=1.7-3
    • libvmod-re2=1.3.1-3

Could someone from traffic take a look at this? I have a feeling (judging from packages in apt) that we're mid upgrade and perhaps beta "just" needs to be upgraded as well?

Related Objects
Search...

Event Timeline

thcipriani created this task.Nov 9 2020, 4:26 PM
Restricted Application added a project: SRE. · View Herald TranscriptNov 9 2020, 4:26 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
thcipriani moved this task from To Triage to Puppet errors on the Beta-Cluster-Infrastructure board.Nov 9 2020, 4:27 PM
thcipriani moved this task from Should be empty (use Release-Engineering-Team) to Watching/External on the Release-Engineering-Team-TODO board.
jijiki triaged this task as Medium priority.Nov 10 2020, 4:06 PM
thcipriani added a comment.Nov 10 2020, 4:16 PM

Even with puppet disabled, packages were upgraded which broke this again. I set the packages to be held for the time being.

root@deployment-cache-text06:~# apt-get install varnish-modules=0.12.1-1+wmf2 libvmod-netmapper=1.7-3 libvmod-re2=1.3.1-3 varnish=5.1.3-1wm15
root@deployment-cache-text06:~# sudo service varnish-frontend restart
root@deployment-cache-text06:~# apt-mark hold varnish-modules=0.12.1-1+wmf2 libvmod-netmapper=1.7-3 libvmod-re2=1.3.1-3 varnish=5.1.3-1wm15
varnish-modules set on hold.
libvmod-netmapper set on hold.
libvmod-re2 set on hold.
varnish set on hold.
thcipriani added a parent task: T267561: Beta needs to be upgraded to Varnish 6.Nov 10 2020, 10:22 PM
hashar closed this task as Declined.Nov 12 2020, 11:27 AM
hashar subscribed.

In T267561 , the varnish related packages have been upgraded to the ones for Varnish 6. Puppet has been disabled in that task. One issue is generating a confd::file which requires some srv_dns parameter.

I think we can just close this task, it is seems entirely redundant with the parent one.

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL