Page MenuHomePhabricator
Create Task
Maniphest T332514

Store state information for the disable tool process outside NFS
Open, In Progress, MediumPublic
Actions

Description

The current process to disable a tool stores state information in plaintext files in the tool's NFS home directory. This is not ideal as it constitutes an unnecessary dependency for NFS which we don't really need, and it could be replaced with some database instead.

Details

SubjectRepoBranchLines +/-
Go back to using the nfs state file to ensure that k8s is disabledcloud/toolforge/disable-toolmaster+11 -3
get_step_complete: fix dereference of rowscloud/toolforge/disable-toolmaster+1 -1
set_step_complete: commit before closing db connectioncloud/toolforge/disable-toolmaster+1 -0
Replace flag files with entries in a databasecloud/toolforge/disable-toolmaster+76 -36
profile::toolforge::disable_tool: include python3-pymysqloperations/puppetproduction+3 -1
profile::toolforge::disable_tool: fix a couple of param namesoperations/puppetproduction+2 -2
Add database config for disable_tool processoperations/puppetproduction+15 -2
Added fake profile::toolforge::disable_tool::disable_tool_db_passwordlabs/privatemaster+1 -0
Customize query in gerrit

Related Objects
Search...

Event Timeline

taavi created this task.Mar 19 2023, 6:32 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMar 19 2023, 6:32 PM
Andrew added a parent task: T170355: Figure out process for deleting an unused tool.Apr 10 2023, 5:11 PM
Andrew added a comment.Apr 10 2023, 11:08 PM

At the time I used NFS because it was reachable from both inside and outside the VM realm. Now it is only inside the VM realm so it doesn't really work for state tracking anyway.

@taavi do you have an opinion about where that database is hosted? It's easiest to do in toolsdb (where I assume there's already a fair bit of meta-tool tracking already.)

bd808 subscribed.Apr 10 2023, 11:23 PM

It's easiest to do in toolsdb (where I assume there's already a fair bit of meta-tool tracking already.)

The tracking for maintain-dbusers is actually in the labsdbaccounts database on the m5 cluster.

Andrew added a comment.Apr 10 2023, 11:45 PM
In T332514#8769755, @bd808 wrote:

It's easiest to do in toolsdb (where I assume there's already a fair bit of meta-tool tracking already.)

The tracking for maintain-dbusers is actually in the labsdbaccounts database on the m5 cluster.

Oh dang! Well, /that/ won't work.

Andrew claimed this task.Apr 11 2023, 9:09 PM
gerritbot added a comment.Apr 11 2023, 9:13 PM

Change 907982 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[labs/private@master] Added fake profile::toolforge::disable_tool::disable_tool_db_password

https://gerrit.wikimedia.org/r/907982

gerritbot added a project: Patch-For-Review.Apr 11 2023, 9:13 PM

Change 907982 merged by Andrew Bogott:

[labs/private@master] Added fake profile::toolforge::disable_tool::disable_tool_db_password

https://gerrit.wikimedia.org/r/907982

Andrew mentioned this in rLPRIa5a01826df9c: Added fake profile::toolforge::disable_tool::disable_tool_db_password.Apr 11 2023, 9:14 PM
gerritbot added a comment.Apr 11 2023, 9:14 PM

Change 907983 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] Add database config for disable_tool process

https://gerrit.wikimedia.org/r/907983

gerritbot added a comment.Apr 11 2023, 10:52 PM

Change 907995 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[cloud/toolforge/disable-tool@master] Replace flag files with entries in a database

https://gerrit.wikimedia.org/r/907995

fnegri subscribed.Apr 12 2023, 12:17 PM
gerritbot added a comment.Apr 12 2023, 4:47 PM

Change 907983 merged by Andrew Bogott:

[operations/puppet@production] Add database config for disable_tool process

https://gerrit.wikimedia.org/r/907983

gerritbot added a comment.Apr 12 2023, 6:18 PM

Change 908317 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] profile::toolforge::disable_tool: fix a couple of param names

https://gerrit.wikimedia.org/r/908317

gerritbot added a comment.Apr 12 2023, 6:20 PM

Change 908317 merged by Andrew Bogott:

[operations/puppet@production] profile::toolforge::disable_tool: fix a couple of param names

https://gerrit.wikimedia.org/r/908317

gerritbot added a comment.Apr 12 2023, 6:39 PM

Change 908319 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] profile::toolforge::disable_tool: include python3-pymysql

https://gerrit.wikimedia.org/r/908319

gerritbot added a comment.Apr 12 2023, 6:41 PM

Change 908319 merged by Andrew Bogott:

[operations/puppet@production] profile::toolforge::disable_tool: include python3-pymysql

https://gerrit.wikimedia.org/r/908319

gerritbot added a comment.Apr 12 2023, 6:43 PM

Change 907995 merged by jenkins-bot:

[cloud/toolforge/disable-tool@master] Replace flag files with entries in a database

https://gerrit.wikimedia.org/r/907995

Andrew mentioned this in rCTDT3a08a9925a81: Replace flag files with entries in a database.Apr 12 2023, 6:45 PM
gerritbot added a comment.Apr 12 2023, 7:02 PM

Change 908324 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[cloud/toolforge/disable-tool@master] set_step_complete: commit before closing db connection

https://gerrit.wikimedia.org/r/908324

gerritbot added a comment.Apr 12 2023, 7:03 PM

Change 908324 merged by jenkins-bot:

[cloud/toolforge/disable-tool@master] set_step_complete: commit before closing db connection

https://gerrit.wikimedia.org/r/908324

Andrew mentioned this in rCTDTbb81aac0ce28: set_step_complete: commit before closing db connection.Apr 12 2023, 7:03 PM
Maintenance_bot removed a project: Patch-For-Review.Apr 12 2023, 7:10 PM
gerritbot added a comment.Apr 12 2023, 7:55 PM

Change 908342 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[cloud/toolforge/disable-tool@master] get_step_complete: fix dereference of rows

https://gerrit.wikimedia.org/r/908342

gerritbot added a project: Patch-For-Review.Apr 12 2023, 7:55 PM
gerritbot added a comment.Apr 12 2023, 8:07 PM

Change 908342 merged by jenkins-bot:

[cloud/toolforge/disable-tool@master] get_step_complete: fix dereference of rows

https://gerrit.wikimedia.org/r/908342

Andrew mentioned this in rCTDT31e857a01995: get_step_complete: fix dereference of rows.Apr 12 2023, 8:08 PM
Maintenance_bot removed a project: Patch-For-Review.Apr 12 2023, 8:10 PM
Andrew mentioned this in T334629: Update maintain_kubeusers to use the toolstate database.Apr 12 2023, 9:33 PM
gerritbot added a comment.Apr 13 2023, 3:38 PM

Change 908586 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[cloud/toolforge/disable-tool@master] Go back to using the nfs state file to ensure that k8s is disabled

https://gerrit.wikimedia.org/r/908586

gerritbot added a project: Patch-For-Review.Apr 13 2023, 3:38 PM
gerritbot added a comment.Apr 13 2023, 3:47 PM

Change 908586 merged by jenkins-bot:

[cloud/toolforge/disable-tool@master] Go back to using the nfs state file to ensure that k8s is disabled

https://gerrit.wikimedia.org/r/908586

Andrew mentioned this in rCTDTb5b676c325b1: Go back to using the nfs state file to ensure that k8s is disabled.Apr 13 2023, 3:48 PM
Maintenance_bot removed a project: Patch-For-Review.Apr 13 2023, 4:10 PM
Andrew added a comment.Apr 13 2023, 9:00 PM

This is now fixed except for the maintain_kubeusers subtask

bd808 moved this task from Backlog to In Progress on the Toolforge board.Apr 27 2023, 11:24 PM
bd808 mentioned this in T336452: Tool deletion process seems to be stuck (multiple tools showing 40+ days disabled).May 10 2023, 10:47 PM
taavi moved this task from In Progress to Ready to be worked on on the Toolforge board.Nov 7 2023, 2:23 PM
Andrew mentioned this in T353351: Provide tools for disabling the grid for specific tools.Dec 13 2023, 4:56 PM
dcaro changed the task status from Open to In Progress.Jan 24 2024, 3:48 PM
dcaro triaged this task as Medium priority.
dcaro moved this task from Ready to be worked on to Workspace for triaging whenever needed on the Toolforge board.
dcaro moved this task from Workspace for triaging whenever needed to Ready to be worked on on the Toolforge board.Feb 21 2024, 4:03 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL