Page MenuHomePhabricator
Create Task
Maniphest T234629

Move the Analytics infrastructure to Debian Buster
Closed, ResolvedPublic0 Estimated Story Points
Actions

Description

Tracking task for upgrading Analytics systems to Debian Buster.

  • Q4 2019/2020
  • Matomo (probably worth to create matomo1002?) - IN PROGRESS
  • Archiva (probably worth to create archiva1002?) - IN PROGRESS
  • Notebooks - easier after T243934, probably we are going to decom them decom tracked in T249752
  • Druid Analytics - First Druid nodes on Buster in T252771 - IN PROGRESS
  • Druid Public
  • Q1 2020/2021
  • Q2 2020/2021
  • Hadoop (needs BigTop 1.5 deployed)
  • Yarn and Hue (analytics-tool1001) - This probably will need Packaging Hue for Python3, see T233073
  • Q3 2020/2021

Relevant task:

  • find a partman recipe able to preserve /srv - T252027
  • new ganeti nodes T228924

Details

SubjectRepoBranchLines +/-
Default PYSPARK_PYTHON to exact versioned python executable used on driver.operations/debs/spark2debian+20 -4
Customize query in gerrit

Related Objects
Search...

Event Timeline

elukey triaged this task as Medium priority.Oct 4 2019, 2:06 PM
elukey created this task.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 4 2019, 2:06 PM
elukey added a subtask: T231067: Install Debian Buster on Hadoop.Oct 4 2019, 2:07 PM
MoritzMuehlenhoff subscribed.Oct 7 2019, 8:06 AM
Milimetric moved this task from Incoming to Operational Excellence on the Analytics board.Oct 7 2019, 4:06 PM
Milimetric added a project: Analytics-Kanban.
Milimetric moved this task from Next Up to Parent Tasks on the Analytics-Kanban board.
elukey mentioned this in T220542: Update R from 3.3.3 to 3.6.0 on stat and notebook machines.Oct 16 2019, 8:12 AM
elukey updated the task description. (Show Details)Feb 18 2020, 12:55 PM
elukey changed the status of subtask T231067: Install Debian Buster on Hadoop from Open to Stalled.Feb 18 2020, 2:36 PM
elukey updated the task description. (Show Details)Feb 18 2020, 2:57 PM
elukey updated the task description. (Show Details)
elukey updated the task description. (Show Details)
nshahquinn-wmf subscribed.Feb 19 2020, 3:13 AM
elukey updated the task description. (Show Details)Feb 20 2020, 7:08 AM
elukey updated the task description. (Show Details)
elukey updated the task description. (Show Details)Feb 28 2020, 2:05 PM
elukey updated the task description. (Show Details)Mar 13 2020, 8:56 AM
elukey updated the task description. (Show Details)May 14 2020, 6:56 AM
elukey updated the task description. (Show Details)May 14 2020, 1:23 PM
elukey updated the task description. (Show Details)May 14 2020, 3:20 PM
elukey added a comment.May 22 2020, 2:05 PM

For the stat100x, Kafka and Druid nodes it would be great that the Partman recipe letf /srv intact, but it doesn't seem feasible at the moment: https://phabricator.wikimedia.org/T252027

In the past we used to remove the partman recipe from netboot.cfg, and drive the debian install manually, but this seems not the best option since once one stops the d-i process then the next steps will belong to the standard debian upstream install process, not ours (for example DNS resolvers config will be missed, etc..).

elukey updated the task description. (Show Details)May 25 2020, 6:11 AM
elukey updated the task description. (Show Details)Jun 3 2020, 2:57 PM
elukey updated the task description. (Show Details)
elukey updated the task description. (Show Details)Jun 3 2020, 3:08 PM
elukey added a comment.EditedJun 4 2020, 9:53 AM

@Ottomata today Miriam asked to me some info about why pyspark on stat100[5,8] were yielding version issues (3.7 on driver vs 3.5 on workers) and I found https://gerrit.wikimedia.org/r/#/c/operations/debs/spark2/+/562651/. Do we have another workaround or should we revamp that patch? I didn't follow the python versioning issues at the time :(

Cc: @EBernhardson @Miriam

elukey added subscribers: Miriam, EBernhardson.Jun 4 2020, 10:01 AM
Ottomata added a comment.EditedJun 4 2020, 2:16 PM

Ah yes! https://phabricator.wikimedia.org/T229347#5439259

PYSPARK_PYTHON=python3.7 pyspark2 --master yarn

I added https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Spark#Set_the_python_version_pyspark_should_use to document.

We should probably change the spark2 deb to automatically set PYSPARK_PYTHON to the default python version on the launching node.

Ping @Miriam

Miriam added a comment.Jun 4 2020, 2:34 PM

Oh this is great, thanks so much @Ottomata !

gerritbot added a comment.Jun 4 2020, 2:52 PM

Change 602386 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/debs/spark2@debian] Default PYSPARK_PYTHON to exact versioned python executable used on driver.

https://gerrit.wikimedia.org/r/602386

gerritbot added a project: Patch-For-Review.Jun 4 2020, 2:52 PM
elukey updated the task description. (Show Details)Jun 5 2020, 10:26 AM
elukey updated the task description. (Show Details)Jun 5 2020, 5:54 PM
• Nuria closed subtask T253980: Upgrade Druid to Debian Buster as Resolved.Jun 8 2020, 7:39 PM
elukey updated the task description. (Show Details)Jun 9 2020, 8:45 AM
elukey updated the task description. (Show Details)Jun 18 2020, 12:32 PM
• Nuria closed subtask T252740: Move Matomo to Debian Buster as Resolved.Jun 19 2020, 4:56 PM
elukey updated the task description. (Show Details)Jul 2 2020, 7:52 AM
gerritbot added a comment.Jul 8 2020, 2:16 PM

Change 602386 merged by Ottomata:
[operations/debs/spark2@debian] Default PYSPARK_PYTHON to exact versioned python executable used on driver.

https://gerrit.wikimedia.org/r/602386

Maintenance_bot removed a project: Patch-For-Review.Jul 8 2020, 3:11 PM
• Nuria closed subtask T252767: Move Archiva to Debian Buster as Resolved.Jul 23 2020, 4:37 AM
elukey updated the task description. (Show Details)Aug 27 2020, 9:49 AM
• Nuria closed subtask T255026: Upgrade schema[12]00[12] to Debian Buster as Resolved.Sep 1 2020, 9:31 PM
elukey updated the task description. (Show Details)Sep 2 2020, 9:43 AM
elukey updated the task description. (Show Details)
• Nuria closed subtask T255123: Upgrade Kafka Brokers to Debian Buster as Resolved.Sep 9 2020, 3:12 PM
klausman claimed this task.Sep 28 2020, 10:12 AM
klausman moved this task from Parent Tasks to Done on the Analytics-Kanban board.
• Nuria moved this task from Done to Parent Tasks on the Analytics-Kanban board.Sep 30 2020, 11:28 PM
elukey updated the task description. (Show Details)Oct 1 2020, 9:01 AM
• fdans closed subtask T255028: Move the stat1004-6-7 hosts to Debian Buster as Resolved.Oct 29 2020, 4:21 PM
Ottomata edited projects, added Analytics-Clusters; removed Analytics.Dec 14 2020, 5:00 PM
Ottomata removed a project: Analytics-Clusters.Jan 4 2021, 4:59 PM
elukey changed the status of subtask T231067: Install Debian Buster on Hadoop from Stalled to Open.Feb 18 2021, 7:29 AM
• razzi closed subtask T231067: Install Debian Buster on Hadoop as Resolved.Jul 28 2021, 6:24 PM
Ottomata reassigned this task from klausman to • razzi.Jul 29 2021, 7:31 PM
Ottomata updated the task description. (Show Details)
Ottomata updated the task description. (Show Details)
Ottomata added a project: Analytics-Clusters.
Ottomata added a subscriber: klausman.
Ottomata removed • razzi as the assignee of this task.Aug 2 2021, 3:40 PM
Ottomata added a subscriber: • razzi.
Ottomata updated the task description. (Show Details)Aug 4 2021, 7:48 PM
Ottomata updated the task description. (Show Details)
elukey mentioned this in T288804: Upgrade the Data Engineering infrastructure to Debian Bullseye.Aug 13 2021, 8:01 AM
herron closed subtask T252773: Move kafkamon hosts to Debian Buster as Resolved.Aug 19 2021, 6:03 PM
klausman unsubscribed.Aug 23 2021, 7:22 AM
BTullis updated the task description. (Show Details)Aug 24 2021, 10:15 AM
Ottomata updated the task description. (Show Details)Oct 18 2021, 3:27 PM
Ottomata moved this task from Backlog to Q2 2021/2022 on the Analytics-Clusters board.Oct 25 2021, 3:38 PM
odimitrijevic subscribed.Oct 27 2021, 10:17 PM

@elukey are you ok with us closing this task given that the Cassandra upgrade is near completion?

elukey closed this task as Resolved.Oct 28 2021, 6:14 AM
elukey claimed this task.

Sure!

BTullis updated the task description. (Show Details)Oct 7 2022, 10:15 AM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits