Page MenuHomePhabricator

Reorganize oozie jobs to not use mobile cache webrequest_source {hawk} [21 pts]
Closed, ResolvedPublic

Description

After the mobile cache's have been merged into text, we should reorganize the oozie jobs to no longer consider the mobile when starting. If no data comes into the mobile source, these jobs will block until something does.

Event Timeline

Ottomata created this task.Dec 30 2015, 3:16 PM
Ottomata raised the priority of this task from to Medium.
Ottomata updated the task description. (Show Details)
Ottomata added a project: Analytics-Kanban.
Ottomata added a subscriber: JAllemandou.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptDec 30 2015, 3:16 PM
Nuria added a subscriber: Nuria.Jan 11 2016, 7:11 PM

There are no jobs running on mobile but we would need to change
in ("mobile", "text")
to
In ("text")

Plus start/stop jobs

BBlack added a subscriber: BBlack.Jan 13 2016, 3:36 PM

The switch of traffic off of the mobile cluster is tentatively scheduled to begin on Tuesday, Jan 19th and take at least several hours. Will update here when it's complete.

Nuria renamed this task from Reorganize oozie jobs to not use mobile cache webrequest_source to Reorganize oozie jobs to not use mobile cache webrequest_source [13].Jan 18 2016, 5:54 PM
Nuria set Security to None.

Be aware that after this change we need to document how to proceed for users doing ad hoc selects here:
https://wikitech.wikimedia.org/wiki/Analytics/Data/Webrequest#Changes_and_known_problems_since_2015-03-04

madhuvishy renamed this task from Reorganize oozie jobs to not use mobile cache webrequest_source [13] to Reorganize oozie jobs to not use mobile cache webrequest_source {hawk} [13 pts].Jan 18 2016, 6:42 PM
Ottomata claimed this task.Jan 18 2016, 9:52 PM
Ottomata moved this task from Next Up to In Progress on the Analytics-Kanban board.

Change 264868 had a related patch set uploaded (by Ottomata):
Use webrequest_source text for AppSessionMetrics, mobile is merging with text

https://gerrit.wikimedia.org/r/264868

Change 264870 had a related patch set uploaded (by Ottomata):
Remove 'mobile' webrequest_source, mobile is merging with text

https://gerrit.wikimedia.org/r/264870

@JAllemandou, just pushed changes to refinery and refinery-source for removing webrequest_source mobile. Please review :)

https://gerrit.wikimedia.org/r/#/c/264868/
https://gerrit.wikimedia.org/r/#/c/264870/

Feel free to make changes, merge, and/or build new refinery-source jars and mvn deploy them if you get around to it before I start working in my morning.

Change 264868 merged by Joal:
Use webrequest_source text for AppSessionMetrics, mobile is merging with text

https://gerrit.wikimedia.org/r/264868

Change 264940 had a related patch set uploaded (by Joal):
Clean repo from mobile partition leftovers

https://gerrit.wikimedia.org/r/264940

Change 264940 merged by Mforns:
Clean repo from mobile partition leftovers

https://gerrit.wikimedia.org/r/264940

Change 264949 had a related patch set uploaded (by Joal):
Modify oozie job to use jars v0.0.25

https://gerrit.wikimedia.org/r/264949

Change 264949 merged by Ottomata:
Modify oozie job to use jars v0.0.25

https://gerrit.wikimedia.org/r/264949

Change 264870 merged by Joal:
Remove mobile webrequest_source merging it in text

https://gerrit.wikimedia.org/r/264870

Change 267230 had a related patch set uploaded (by BBlack):
eqiad: remove last cache_mobile frontend

https://gerrit.wikimedia.org/r/267230

BBlack added a comment.Feb 2 2016, 1:34 PM

Just to be clear: no real user traffic is flowing in webrequest_mobile, but there's still some internal healthcheck traffic keeping the data from dropping off completely. We're blocking now on this oozie change before we can continue with the decom, please fire when ready.

Thanks for the clarification @BBlack, we plan on working on moving our oozie stuff starting today with @Ottomata.

Change 267889 had a related patch set uploaded (by Ottomata):
Revert "Revert "Use webrequest_source text for AppSessionMetrics, mobile is merging with text""

https://gerrit.wikimedia.org/r/267889

Change 267891 had a related patch set uploaded (by Ottomata):
Revert "Revert "Remove mobile webrequest_source merging it in text""

https://gerrit.wikimedia.org/r/267891

Change 267889 merged by Joal:
Revert "Revert "Use webrequest_source text for AppSessionMetrics, mobile is merging with text""

https://gerrit.wikimedia.org/r/267889

Change 267891 merged by Ottomata:
Revert "Revert "Remove mobile webrequest_source merging it in text""

https://gerrit.wikimedia.org/r/267891

Change 268142 had a related patch set uploaded (by Ottomata):
Remove webrequest_mobile from list of webrequest topics that camus imports

https://gerrit.wikimedia.org/r/268142

Change 268142 merged by Ottomata:
Remove webrequest_mobile from list of webrequest topics that camus imports

https://gerrit.wikimedia.org/r/268142

We are mostly done! Our process was here:
https://etherpad.wikimedia.org/p/refinery_mobile2text

We still need to wait for some data to move around in HDFS before we can relaunch a few jobs. But, there are no more running jobs that depend on webrequest_mobile, so @BBlack you can go ahead and start the process of turning off all mobile cache traffic.

Change 267230 merged by BBlack:
eqiad: remove last cache_mobile frontend

https://gerrit.wikimedia.org/r/267230

Ottomata moved this task from Paused to In Progress on the Analytics-Kanban board.Feb 3 2016, 5:01 PM
JAllemandou renamed this task from Reorganize oozie jobs to not use mobile cache webrequest_source {hawk} [13 pts] to Reorganize oozie jobs to not use mobile cache webrequest_source {hawk} [21 pts].Feb 5 2016, 10:53 AM
JAllemandou moved this task from In Progress to In Code Review on the Analytics-Kanban board.
JAllemandou moved this task from In Code Review to Done on the Analytics-Kanban board.
Nuria closed this task as Resolved.Feb 9 2016, 8:48 PM