Page MenuHomePhabricator
Create Task
Maniphest T130031

Cleanup of the project stewardbots
Open, Needs TriagePublic
Actions

Description

  • Currently there's a lot of files, folders and a lot of stuff from when the tools were run in the Toolserver. We should identify which ones are still for use and updated, and delete the others from Labs.
  • Also, a better organization would be much appreciated.

To review (referenced to $HOME/)

  • .cache: From 2019, needed?
  • .cloud-locale-test.skip: Empty, needed?
  • core: What is this?
  • docs: Old Toolserver documentation, can be deleted IMHO.
  • .composer - maybe needed by the Elections tool?
  • config: ???
  • .python27_compiled: No longer required I guess.
  • StewardElectionsBot and Watchlist-bots: most likely broken (py2, freenode) and replaced by other bots. Suggest deleting.
  • .subversion: old SVN stuff, can be deleted I guess.

Details

SubjectRepoBranchLines +/-
Remove stewardbots-ui directorylabs/tools/stewardbotsmaster+4 -226
Follow-up cdd4595: Update index.php (legacy webservice)labs/tools/stewardbotsmaster+6 -24
Delete privacy.htmllabs/tools/stewardbotsmaster+0 -34
Delete cron.txtlabs/tools/stewardbotsmaster+0 -4
Fix tox issues for stewardbotslabs/tools/stewardbotsmaster+8 -5
Customize query in gerrit

Related Objects
Search...

Event Timeline

MarcoAurelio created this task.Mar 15 2016, 5:04 PM
Restricted Application added subscribers: JEumerus, Aklapper. · View Herald TranscriptMar 15 2016, 5:04 PM
MarcoAurelio updated the task description. (Show Details)Mar 15 2016, 5:05 PM
Luke081515 renamed this task from Cleanup of the project to Cleanup of the project stewardbots.Mar 15 2016, 7:54 PM
MarcoAurelio awarded a token.Apr 4 2016, 5:18 PM
MarcoAurelio updated the task description. (Show Details)Sep 1 2016, 12:58 PM
MarcoAurelio triaged this task as Low priority.Sep 1 2016, 1:02 PM
MarcoAurelio rescinded a token.
Shanmugamp7 subscribed.Sep 1 2016, 3:35 PM
Trijnstel subscribed.Oct 11 2016, 5:45 PM
Matanya moved this task from Untriaged to Medium priority on the Stewards-and-global-tools board.Apr 19 2017, 8:00 PM
MarcoAurelio added a project: Essential-Work.Jun 1 2017, 7:53 AM
MarcoAurelio raised the priority of this task from Low to Medium.Jan 26 2020, 10:48 AM
MarcoAurelio added a comment.Jan 29 2020, 11:28 AM

Over the years this has become such a mess... We need to follow proper project structure, etc.

MarcoAurelio added a subtask: T246702: Make tools.stewardbot look clear to a newbie.Mar 4 2020, 7:02 PM
Zabe subscribed.Oct 4 2021, 11:45 PM
TheresNoTime claimed this task.May 12 2022, 3:24 PM
TheresNoTime subscribed.

In doing some patch reviews, I started to go down the yak shaving path, so will grab this to work on in my spare time 😹

TheresNoTime changed the status of subtask T308256: Split stewardbots-ui into own repository from Open to In Progress.May 12 2022, 3:46 PM
TheresNoTime closed subtask T308256: Split stewardbots-ui into own repository as Resolved.May 13 2022, 9:26 PM
Operator873 subscribed.May 13 2022, 9:58 PM
TheresNoTime lowered the priority of this task from Medium to Low.Jul 5 2022, 10:49 AM
TheresNoTime removed TheresNoTime as the assignee of this task.Nov 2 2022, 5:23 PM
TheresNoTime raised the priority of this task from Low to Needs Triage.
MarcoAurelio added a comment.EditedFeb 18 2023, 4:04 PM

Taking a look today, February 18, 2023 I see among other things:

  • a bin folder with symlinks to old bash scripts to restart the bots via Grid, probably no longer needed
  • bigbrother files which, according to Toolforge are deprecated and probably no longer needed since we deploy our bots via k8s.
  • DATA.crontab and DATA.olduser which probably can go too
    • crontab -l lists some deprecated jobs which I think can go too because we use k8s now; tempted to just crontab -r
    • we may wish to set a logrotate on the Grid later nonetheless for the logs that get generated just in case?
  • a core file of 2.8M -> no idea what is this: tailing it just prints an unreadable output
  • 5 venvs - do we need them all?
  • .python27_compiled -- probably not needed either
  • a .subversion dir which doesn't seem to be used?
  • a .hhvm.hhbc file of 4M from 2020
  • old WatchList bots that won't probably work now
  • several other hidden files which need to be evaluated
Stashbot added a comment.Feb 18 2023, 4:09 PM

Mentioned in SAL (#wikimedia-cloud) [2023-02-18T16:09:43Z] <wm-bot> <maurelio> Deleted crontab for stewardbots; backup saved at crontab_20230218.txt | T130031

MarcoAurelio added a comment.EditedFeb 18 2023, 4:13 PM

crontab deleted but saved its contents in a txt file just in case. AFAICS these were the old bigbrother jobs we used to keep the bots active which predate k8s and are no longer needed - checking https://grid-deprecation.toolforge.org/t/stewardbots I see no grid jobs for the last 7 days for our project

gerritbot added a comment.Feb 18 2023, 4:45 PM

Change 889617 had a related patch set uploaded (by MarcoAurelio; author: MarcoAurelio):

[labs/tools/stewardbots@master] Delete cron.txt

https://gerrit.wikimedia.org/r/889617

gerritbot added a project: Patch-For-Review.Feb 18 2023, 4:45 PM
gerritbot added a comment.Feb 18 2023, 4:53 PM

Change 889618 had a related patch set uploaded (by MarcoAurelio; author: MarcoAurelio):

[labs/tools/stewardbots@master] Delete privacy.html

https://gerrit.wikimedia.org/r/889618

gerritbot added a comment.Feb 18 2023, 5:06 PM

Change 890163 had a related patch set uploaded (by MarcoAurelio; author: MarcoAurelio):

[labs/tools/stewardbots@master] Fix tox issues for StewardBot.py

https://gerrit.wikimedia.org/r/890163

MarcoAurelio mentioned this in T246702: Make tools.stewardbot look clear to a newbie.Feb 18 2023, 5:55 PM
gerritbot added a comment.Feb 18 2023, 6:48 PM

Change 890163 merged by jenkins-bot:

[labs/tools/stewardbots@master] Fix tox issues for stewardbots

https://gerrit.wikimedia.org/r/890163

gerritbot added a comment.Feb 18 2023, 6:48 PM

Change 889617 merged by jenkins-bot:

[labs/tools/stewardbots@master] Delete cron.txt

https://gerrit.wikimedia.org/r/889617

gerritbot added a comment.Feb 18 2023, 6:49 PM

Change 889618 merged by jenkins-bot:

[labs/tools/stewardbots@master] Delete privacy.html

https://gerrit.wikimedia.org/r/889618

Maintenance_bot removed a project: Patch-For-Review.Feb 18 2023, 7:10 PM
gerritbot added a comment.Feb 19 2023, 4:00 PM

Change 890188 had a related patch set uploaded (by MarcoAurelio; author: MarcoAurelio):

[labs/tools/stewardbots@master] Follow-up cdd4595: Update index.php (legacy webservice)

https://gerrit.wikimedia.org/r/890188

gerritbot added a project: Patch-For-Review.Feb 19 2023, 4:00 PM
Stashbot added a comment.Feb 19 2023, 4:17 PM

Mentioned in SAL (#wikimedia-cloud) [2023-02-19T16:17:29Z] <wm-bot> <maurelio> rm -v .bigbrotherrc.deprecated # T130031

Stashbot added a comment.Feb 19 2023, 4:20 PM

Mentioned in SAL (#wikimedia-cloud) [2023-02-19T16:20:17Z] <wm-bot> <maurelio> rm -v ...DATA.crontab ...DATA.olduser # cleanup: T130031

MarcoAurelio added a comment.Feb 19 2023, 5:24 PM

More unused/legacy/unneeded files deleted:

# bigbrother leftovers
removed 'bigbrother.sh'

# bin folder (old grid scripts)
tools.stewardbots@tools-sgebastion-10:~/bin$ ls -la
total 16
drwxr-sr-x  2 tools.stewardbots tools.stewardbots 4096 Jul  2  2020 .
drwxrwsr-x 26 tools.stewardbots tools.stewardbots 4096 Feb 19 17:19 ..
lrwxrwxrwx  1 tools.stewardbots tools.stewardbots   70 Jul  2  2020 restart_stewardbot.sh -> /data/project/stewardbots/stewardbots/StewardBot/restart_stewardbot.sh
lrwxrwxrwx  1 tools.stewardbots tools.stewardbots   70 Jul  2  2020 restart_SULWatcher.sh -> /data/project/stewardbots/stewardbots/SULWatcher/restart_SULWatcher.sh

tools.stewardbots@tools-sgebastion-10:~$ rm -Rv bin
removed 'bin/restart_stewardbot.sh'
removed 'bin/restart_SULWatcher.sh'
removed directory 'bin'
MarcoAurelio updated the task description. (Show Details)Feb 19 2023, 5:42 PM
MarcoAurelio updated the task description. (Show Details)Feb 19 2023, 5:48 PM
MarcoAurelio updated the task description. (Show Details)
gerritbot added a comment.Mar 1 2023, 6:23 PM

Change 890188 merged by jenkins-bot:

[labs/tools/stewardbots@master] Follow-up cdd4595: Update index.php (legacy webservice)

https://gerrit.wikimedia.org/r/890188

Maintenance_bot removed a project: Patch-For-Review.Mar 1 2023, 6:31 PM
Aklapper closed subtask T246702: Make tools.stewardbot look clear to a newbie as Resolved.Apr 12 2023, 11:40 AM
gerritbot added a comment.Jul 3 2023, 7:29 PM

Change 935154 had a related patch set uploaded (by AntiCompositeNumber; author: AntiCompositeNumber):

[labs/tools/stewardbots@master] Remove stewardbots-ui directory

https://gerrit.wikimedia.org/r/935154

gerritbot added a project: Patch-For-Review.Jul 3 2023, 7:29 PM
gerritbot added a comment.Jul 4 2023, 10:26 AM

Change 935154 merged by jenkins-bot:

[labs/tools/stewardbots@master] Remove stewardbots-ui directory

https://gerrit.wikimedia.org/r/935154

Maintenance_bot removed a project: Patch-For-Review.Jul 4 2023, 10:30 AM
Stashbot added a comment.Apr 24 2024, 7:26 PM

Mentioned in SAL (#wikimedia-cloud) [2024-04-24T19:26:29Z] <wmbot~anticomposite@tools-bastion-13> rm -r venv/ venv-k8s-py37/ venv-py3/ venv-py37-k8s/ # Remove old unused python 2.7, 3.5, and 3.7 venvs (T130031)

AntiCompositeNumber updated the task description. (Show Details)Apr 24 2024, 7:27 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL