Page MenuHomePhabricator
Create Task
Maniphest T97294

Turn off webrequest udp2log instances.
Closed, ResolvedPublic
Actions

Description

We are ready! Let's do this

Details

SubjectRepoBranchLines +/-
Clean up some unused udp2log stuffoperations/puppetproduction+1 -173
Remove gadolinium related udp2log and varnishncsa stuffoperations/puppetproduction+7 -10
Remove more erbium related udp2log stuffoperations/puppetproduction+2 -30
Set up kafkatee instance on erbium to output fundraising logsoperations/puppetproduction+92 -2
Remove oxygen udp2log related codeoperations/puppetproduction+11 -80
Turn off oxygen's udp2log instance, move 5xx error log to erbium, turn off some erbium filtersoperations/puppetproduction+6 -12
Customize query in gerrit

Related Objects
Search...

Event Timeline

Ottomata created this task.Apr 27 2015, 2:52 PM
Ottomata claimed this task.
Ottomata raised the priority of this task from to Needs Triage.
Ottomata updated the task description. (Show Details)
Ottomata added projects: Analytics-Clusters, acl*sre-team.
Ottomata subscribed.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 27 2015, 2:52 PM
Ottomata added a parent task: T96616: Set up ops kafkatee instance as part of udp2log transition.Apr 27 2015, 2:53 PM
gerritbot subscribed.Apr 27 2015, 2:56 PM

Change 206810 had a related patch set uploaded (by Ottomata):
Turn off oxygen's udp2log instance, move 5xx error log to erbium, turn off some erbium filters

https://gerrit.wikimedia.org/r/206810

gerritbot added a project: Patch-For-Review.Apr 27 2015, 2:56 PM
gerritbot added a comment.Apr 27 2015, 3:15 PM

Change 206810 merged by Ottomata:
Turn off oxygen's udp2log instance, move 5xx error log to erbium, turn off some erbium filters

https://gerrit.wikimedia.org/r/206810

Ottomata mentioned this in rOPUP2977db23668b: Turn off oxygen's udp2log instance, move 5xx error log to erbium, turn off….Apr 27 2015, 3:15 PM
gerritbot added a comment.Apr 27 2015, 3:29 PM

Change 206815 had a related patch set uploaded (by Ottomata):
Remove oxygen udp2log related code

https://gerrit.wikimedia.org/r/206815

gerritbot added a comment.Apr 27 2015, 3:53 PM

Change 206815 merged by Ottomata:
Remove oxygen udp2log related code

https://gerrit.wikimedia.org/r/206815

Ottomata mentioned this in rOPUP9d23fbb695f4: Remove oxygen udp2log related code.Apr 27 2015, 3:53 PM
Ottomata added a comment.Apr 27 2015, 5:15 PM

Today I turned of most udp2log webrequest filters. For now, I have left the Fundraising filters, as well as the 5xx and sampled-1000 filters running. All of these filters are now running on erbium. oxygen's udp2log instance has been shut off. I plan on reinstalling oxygen as Jessie this week, and then will set up an operations kafkatee instance to generate 5xx and sampled logs there. These will be used for operational issue troubleshooting, and will not be synced to stat1002. https://phabricator.wikimedia.org/T96616 will track this.

• ezachte subscribed.Apr 27 2015, 5:22 PM
gerritbot added a comment.Apr 30 2015, 5:59 PM

Change 207858 had a related patch set uploaded (by Ottomata):
Set up kafkatee instance on erbium to output fundraising logs

https://gerrit.wikimedia.org/r/207858

gerritbot added a comment.Apr 30 2015, 6:00 PM

Change 207858 merged by Ottomata:
Set up kafkatee instance on erbium to output fundraising logs

https://gerrit.wikimedia.org/r/207858

Ottomata mentioned this in rOPUP4950aeb08c07: Set up kafkatee instance on erbium to output fundraising logs.Apr 30 2015, 6:01 PM
Ottomata closed subtask T97771: Backport? and install kafkacat (on stat1002?) as Resolved.May 27 2015, 4:22 PM
faidon added a parent task: T111653: Encrypt all the things.Sep 6 2015, 8:59 PM
Jgreen added a subtask: Restricted Task.Sep 16 2015, 5:19 PM
Restricted Application added a subscriber: Matanya. · View Herald TranscriptSep 16 2015, 5:19 PM
Jgreen changed the status of subtask Restricted Task from Open to Stalled.Oct 5 2015, 6:11 PM
Jgreen removed a subtask: T97676: Verify kafkatee use for fundraising logs on erbium.Nov 6 2015, 6:03 PM
Restricted Application added a subscriber: StudiesWorld. · View Herald TranscriptNov 6 2015, 6:03 PM
Ottomata triaged this task as Medium priority.Nov 13 2015, 5:38 PM
Ottomata added a subtask: T117727: Turn off sqstat udp2log instance.
Ottomata added a project: Analytics-Kanban.
Ottomata set Security to None.

Let's turn of the udp2log instance on erbium. @Jeff_Green, if you say this ok, we will do it!

gerritbot added a comment.Nov 13 2015, 5:40 PM

Change 252961 had a related patch set uploaded (by Faidon Liambotis):
Remove role::logging::udp2log::erbium and friends

https://gerrit.wikimedia.org/r/252961

Jgreen subscribed.Nov 13 2015, 5:41 PM
In T97294#1804032, @Ottomata wrote:

Let's turn of the udp2log instance on erbium. @Jeff_Green, if you say this ok, we will do it!

Do it!

Jgreen closed subtask Restricted Task as Resolved.Nov 13 2015, 6:34 PM
gerritbot added a comment.Nov 23 2015, 2:49 PM

Change 254860 had a related patch set uploaded (by Ottomata):
Remove more erbium related udp2log stuff

https://gerrit.wikimedia.org/r/254860

gerritbot added a comment.Nov 23 2015, 2:50 PM

Change 254860 merged by Ottomata:
Remove more erbium related udp2log stuff

https://gerrit.wikimedia.org/r/254860

gerritbot added a comment.Nov 23 2015, 2:59 PM

Change 254864 had a related patch set uploaded (by Ottomata):
Remove gadolinium related udp2log and varnishncsa stuff

https://gerrit.wikimedia.org/r/254864

Ottomata moved this task from Next Up to In Progress on the Analytics-Kanban board.Nov 23 2015, 5:10 PM
Ottomata moved this task from In Progress to Paused on the Analytics-Kanban board.Dec 10 2015, 5:05 PM
gerritbot added a comment.Dec 14 2015, 3:44 PM

Change 254864 merged by Ottomata:
Remove gadolinium related udp2log and varnishncsa stuff

https://gerrit.wikimedia.org/r/254864

Ottomata moved this task from Paused to In Progress on the Analytics-Kanban board.Dec 14 2015, 5:04 PM
gerritbot added a comment.Dec 14 2015, 7:32 PM

Change 259051 had a related patch set uploaded (by Ottomata):
Clean up some unused udp2log stuff

https://gerrit.wikimedia.org/r/259051

gerritbot added a comment.Dec 14 2015, 7:40 PM

Change 259051 merged by Ottomata:
Clean up some unused udp2log stuff

https://gerrit.wikimedia.org/r/259051

Ottomata moved this task from In Progress to Done on the Analytics-Kanban board.Dec 15 2015, 5:01 PM
Ottomata closed this task as Resolved.Dec 15 2015, 7:20 PM
Ottomata closed subtask T117727: Turn off sqstat udp2log instance as Resolved.
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits