Page MenuHomePhabricator

Implementation and testing of glent m0 integration with wmf infra
Closed, ResolvedPublic

Description

Reimplement hive UDF from analytics-hive in spark:

  • get_main_search_request

Write tests for the handshake between wmf analytics infrastructure and glent

  • initSpark
  • readSparkDataset
  • writeDf
  • flattenWmfLog

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMar 12 2019, 10:11 PM
Restricted Application added a project: Analytics. · View Herald TranscriptMar 12 2019, 10:12 PM
EBernhardson renamed this task from Impleentation and testing of glent m0 integration with wmf infra to Implementation and testing of glent m0 integration with wmf infra.Mar 12 2019, 10:22 PM
EBernhardson updated the task description. (Show Details)

Change 496361 had a related patch set uploaded (by EBernhardson; owner: EBernhardson):
[search/glent@master] Reimplement GetMainSearchRequest UDF from refinery-hive

https://gerrit.wikimedia.org/r/496361

Change 496361 merged by jenkins-bot:
[search/glent@master] Reimplement GetMainSearchRequest UDF from refinery-hive

https://gerrit.wikimedia.org/r/496361

Milimetric moved this task from Incoming to Radar on the Analytics board.Mar 18 2019, 3:48 PM
debt closed this task as Resolved.Apr 5 2019, 10:33 PM
Aklapper edited projects, added Analytics-Radar; removed Analytics.Jun 10 2020, 6:44 AM