Page MenuHomePhabricator
Create Task
Maniphest T199271

Upgrade the tools gridengine system
Closed, ResolvedPublic
Actions

Description

In order to remove dependencies on very old versions of gridengine and Ubuntu Trusty, we aim to upgrade to Debian Stretch and SonOfGridengine. There are many pieces to this puzzle. This task is meant to be an overall tracking and gathering point for the effort and it's parts.

Related Objects
Search...

StatusSubtypeAssignedTask
 Resolved• BstormT199271 Upgrade the tools gridengine system
Restricted Task
 Resolved• BstormT213656 Update documentation or config for new grid around adminhost-only commands
 ResolvedtaaviT214427 Enable puppetdb in toolforge
 Resolved• BstormT162955 rebuild tools-grid-master as a large instance
 Resolvedbd808T167412 host-vmem.erb is doing operations that make no sense
 ResolvedNoneT161898 Tools instances flapping puppet failure alerts
 ResolvedNoneT163208 wsexport tool writing output to $HOME/tool/temp puts load on Tool Labs NFS server
 ResolvedFluffT162978 perfectbot runs ListSpeedyDeletions job once per minute
 ResolvedFNDET162979 request tool runs secWatch job once per minute
 ResolvedKalanT163075 ruarbcom tool runs count.py job once per minute
 ResolvedKanashimiT163572 cewbot using '* * * * *' cron that could be replaced with .bigbrotherrc
 ResolvedtaaviT88237 Track and alert based on gridengine error states
 Resolved• BstormT88711 Fully puppetize Grid Engine
 Resolved• BstormT88712 Puppetize adding new node to OGE
 Resolved• BstormT88713 Puppetize adding a host to a particular queue
 DeclinedNoneT88733 Document our GridEngine set up
 Resolved• BstormT95747 Puppetize gridengine master configuration
 ResolvedscfcT107821 Puppetize that h_vmem is a consumable resource
 ResolvedcorenT109316 sge master not starting up on tools-master
 Resolved• BstormT199276 Test running a stretch exec node in the existing system on toolsbeta
 DeclinedNoneT177850 Page if the grid engine master is unreachable
 ResolvedscfcT127388 gridengine master dependencies are missing for gridengine_resources
 Resolved• BstormT200557 Create a stretch and Son of Grid Engine grid in toolsbeta
 ResolvedaborreroT207970 toolforge: add misctools and jobutils packages to stretch
 ResolvedhasharT208873 Add python3 to the containers that run in CI for puppet
 Declined• BstormT209627 Kubernetes and docker packages for stretch are needed for toolforge bastions
 ResolvedaborreroT210098 Port cgroup restrictions and definitions to systemd/stretch for bastions
 ResolvedaborreroT215154 Toolforge: sgebastion: systemd resource control not working
 InvalidNoneT218720 Add a blkio restriction to bastion cgroups
 Resolved• BstormT211055 Fix the init script for gridmaster in sonofgridengine
 Resolved• BstormT211258 Get shadow master to actually work in toolsbeta
 Resolved• BstormT212153 Stand up the new sonofgridengine and stretch grid for testing in toolforge
 Resolved• BstormT212390 Basic lighttpd+php webservice fails to run on Stretch grid
 Resolved• BstormT168701 Rotate Grid Engine accounting file
 Resolved• BstormT212981 npm is missing from Stretch image
 DuplicateNoneT196495 Limit ability of a single user/tool to overwhealm job grid
 Resolved• BstormT123270 Make gridengine exec hosts also submit hosts
 Resolved• BstormT67777 Limit number of jobs users can execute in parallel
 Resolved• BstormT213183 Set up puppet to handle the global and scheduler configuration of gridengine
 DeclinedMusikAnimalT216268 Custom ruby interpreter compiled on Trusty can't find libssl.so.1.0.0 on Stretch
 ResolvedAz1568T216314 loltrs crashes upon migrating to stretch grid
 Resolvedbd808T215963 Stretch grid problem: python 'oursql' package fails to compile with MariaDB client libraries
 ResolvedAz1568T216370 IP address list for grid nodes / Freenode iline request
 DeclinedNoneT151603 Grid jobs often stuck after Tool Labs maintenance

Event Timeline

• Bstorm created this task.Jul 10 2018, 8:01 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJul 10 2018, 8:01 PM
• Bstorm added subtasks: Restricted Task, T162955: rebuild tools-grid-master as a large instance.Jul 10 2018, 8:03 PM
• Bstorm added a subtask: T161898: Tools instances flapping puppet failure alerts.Jul 10 2018, 8:08 PM
• Bstorm added a subtask: T88237: Track and alert based on gridengine error states.
• Bstorm added a subtask: T88733: Document our GridEngine set up.
• Bstorm added a subtask: T88711: Fully puppetize Grid Engine.Jul 10 2018, 8:10 PM
bd808 triaged this task as Medium priority.Jul 10 2018, 8:14 PM
bd808 added projects: Epic, Toolforge.
bd808 subscribed.
• Bstorm added a subtask: T177850: Page if the grid engine master is unreachable.Jul 11 2018, 4:51 PM
bd808 added a subtask: T127388: gridengine master dependencies are missing for gridengine_resources.Jul 12 2018, 11:47 PM
bd808 closed subtask T127388: gridengine master dependencies are missing for gridengine_resources as Resolved.
zhuyifei1999 subscribed.Jul 19 2018, 5:12 AM
• Bstorm closed subtask T199276: Test running a stretch exec node in the existing system on toolsbeta as Resolved.Jul 27 2018, 6:51 PM
MGChecker subscribed.Sep 19 2018, 11:24 PM
• Bstorm added a parent task: T186029: Remove support for Ubuntu Trusty prior to upstream End Of Life for release.Oct 12 2018, 10:05 PM
bd808 added a project: Cloud-VPS (Ubuntu Trusty Deprecation).Oct 12 2018, 10:20 PM
bd808 moved this task from Backlog to Toolforge infra on the Cloud-VPS (Ubuntu Trusty Deprecation) board.
• Bstorm moved this task from Inbox to Epics on the cloud-services-team (Kanban) board.Oct 18 2018, 4:46 PM
bd808 mentioned this in T132880: tools.jembot PHP processes run out of memory and leave orphan php-cgi processes regularly.Oct 29 2018, 4:58 PM
bd808 mentioned this in T208711: Update the "tesseract-ben" package on Toolforge for OCR on Bengali Wikisource.Nov 29 2018, 12:45 AM
• Bstorm closed subtask T200557: Create a stretch and Son of Grid Engine grid in toolsbeta as Resolved.Dec 19 2018, 3:35 PM
bd808 added a parent task: T187219: Remove support for Trusty Grid Engine exec hosts.Dec 19 2018, 10:31 PM
bd808 removed a parent task: T186029: Remove support for Ubuntu Trusty prior to upstream End Of Life for release.
bd808 added a subtask: T168701: Rotate Grid Engine accounting file.Jan 4 2019, 10:58 PM
bd808 moved this task from Backlog to In Progress on the Toolforge board.Jan 5 2019, 5:13 PM
bd808 added subtasks: T196495: Limit ability of a single user/tool to overwhealm job grid, T67777: Limit number of jobs users can execute in parallel.Jan 5 2019, 5:20 PM
bd808 added a subtask: T123270: Make gridengine exec hosts also submit hosts.Jan 8 2019, 4:06 AM
bd808 removed a subtask: T67777: Limit number of jobs users can execute in parallel.
bd808 closed subtask T212153: Stand up the new sonofgridengine and stretch grid for testing in toolforge as Resolved.Jan 9 2019, 5:19 PM
bd808 closed subtask T162955: rebuild tools-grid-master as a large instance as Resolved.Jan 9 2019, 5:26 PM
• GTirloni closed subtask T161898: Tools instances flapping puppet failure alerts as Resolved.Jan 10 2019, 12:03 PM
• Bstorm closed subtask T123270: Make gridengine exec hosts also submit hosts as Resolved.Jan 18 2019, 1:14 AM
• Bstorm closed subtask T212981: npm is missing from Stretch image as Resolved.Jan 23 2019, 7:38 PM
bd808 closed subtask T168701: Rotate Grid Engine accounting file as Resolved.Jan 24 2019, 4:13 PM
bd808 added subtasks: T216268: Custom ruby interpreter compiled on Trusty can't find libssl.so.1.0.0 on Stretch, T216314: loltrs crashes upon migrating to stretch grid, T215963: Stretch grid problem: python 'oursql' package fails to compile with MariaDB client libraries, T216370: IP address list for grid nodes / Freenode iline request.Feb 20 2019, 5:28 PM
bd808 changed the status of subtask T215963: Stretch grid problem: python 'oursql' package fails to compile with MariaDB client libraries from Open to Stalled.Feb 20 2019, 10:40 PM
Az1568 closed subtask T216314: loltrs crashes upon migrating to stretch grid as Resolved.Feb 24 2019, 10:09 PM
Legoktm closed subtask T216370: IP address list for grid nodes / Freenode iline request as Resolved.Feb 26 2019, 6:16 PM
MusikAnimal closed subtask T216268: Custom ruby interpreter compiled on Trusty can't find libssl.so.1.0.0 on Stretch as Resolved.Feb 27 2019, 1:05 AM
bd808 changed the status of subtask T216268: Custom ruby interpreter compiled on Trusty can't find libssl.so.1.0.0 on Stretch from Resolved to Declined.Feb 27 2019, 1:08 AM
bd808 mentioned this in R1922:1326fa78a64d: Remove Trusty grid status.Mar 25 2019, 3:37 AM
bd808 mentioned this in T176897: Update Bastion/Grid Engine to PHP 5.6.Mar 26 2019, 12:29 AM
bd808 added a subtask: T151603: Grid jobs often stuck after Tool Labs maintenance.Mar 26 2019, 12:49 AM
bd808 removed a parent task: T187219: Remove support for Trusty Grid Engine exec hosts.Mar 27 2019, 11:29 PM
• Bstorm closed this task as Resolved.Jun 25 2019, 4:50 PM
• Bstorm closed subtask T177850: Page if the grid engine master is unreachable as Declined.Oct 23 2019, 3:34 PM
• Bstorm closed subtask T88711: Fully puppetize Grid Engine as Resolved.Nov 27 2019, 4:32 PM
bd808 closed subtask T151603: Grid jobs often stuck after Tool Labs maintenance as Declined.Jan 4 2020, 9:19 PM
Legoktm closed subtask T215963: Stretch grid problem: python 'oursql' package fails to compile with MariaDB client libraries as Resolved.Feb 9 2020, 8:11 AM
MGChecker unsubscribed.Feb 9 2020, 1:09 PM
taavi closed subtask T88237: Track and alert based on gridengine error states as Resolved.Oct 10 2021, 3:50 PM
dcaro closed subtask T88733: Document our GridEngine set up as Declined.Jan 24 2024, 3:26 PM
taavi closed subtask Restricted Task as Declined.Mar 12 2024, 1:30 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits