Page MenuHomePhabricator
Create Task
Maniphest T316232

[tbs.harbor.toolsbeta] Create a postgres DB with trove and plug the current instance there
Closed, ResolvedPublic2 Estimated Story Points
Actions

Description

  • Instead of using the docker container that comes with docker-compose by default:
    • create a new database using trove for the project
    • plug that one into the docker-compose setup by changing the hiera settings in horizon (under the project puppet config)
    • change the db password of the new db to match the old one
    • test it all works
  • Document also where it needs to be configured in the README for the buildservice repository:

https://github.com/toolforge/buildservice

So then we can use that to create the puppet manifests.

  • remove the (now) unused harbordb instances and cleanup the hiera harbordb puppet config (in the horizon UI, remove the prefix specific config)
  • remove the (now) unused harbor-* instances (we only use harborweb) and the associated project prefix config (same as above)

On how to create a trove DB:
https://wikitech.wikimedia.org/wiki/Help:Adding_a_Database_to_a_Cloud_VPS_Project#Trove:_Database_as_a_Service_for_Cloud_VPS

Related Objects
Search...

StatusSubtypeAssignedTask
 StalledLucasWerkmeisterT320140 Migrate wd-shex-infer from Toolforge GridEngine to Toolforge Kubernetes
 ResolvedmatmarexT319707 Migrate dtcheck from Toolforge GridEngine to Toolforge Kubernetes
 ResolvedLegoktmT320062 Migrate steve-adder from Toolforge GridEngine to Toolforge Kubernetes
 ResolvedLegoktmT320011 Migrate rfa-voting-history from Toolforge GridEngine to Toolforge Kubernetes
 OpendcaroT194332 [Epic] Make Toolforge a proper platform as a service with push-to-deploy and build packs
 In ProgressdcaroT267374 [tbs.beta] Create a toolforge build service beta release
 ResolveddcaroT316541 [tbs.harbor] Rebuild toolsbeta-harborweb-2 with debian 11(bullseye)
 ResolvedRaymond_NdibeT324827 tbs: user-story 3 - I want to cancel the build using the toolforge build cancel feature
 ResolvedRaymond_NdibeT324828 tbs: user-story 3 - cancel cli should give the right error when the service is down
 ResolvedRaymond_NdibeT331409 [tbs.cli] The cancel command should return a message saying that the build is already finished when trying to cancel a finished build
 ResolvedRaymond_NdibeT331410 [tbs.cli] The cancel command should show a useful message when the connection to tekton works but returns http error (requests.exceptions.HTTPError)
 ResolvedNoneT332294 [tbs][cli] build cancel --help lists the -v option twice
 ResolveddcaroT324833 tbs: user-story 5 - I want to check the status of a given build using the toolforge build show feature.
 ResolveddcaroT324830 tbs: user-story 4 - I want to check the status of the last build using the toolforge build show feature
 ResolvedRaymond_NdibeT324831 tbs: user-story 4 - show cli should give the right error when the service is down
 ResolveddcaroT324823 tbs: user-story 2 - Feature: I can start a new build from a git url
 ResolvedRaymond_NdibeT323555 [tbs][cli] Implement the global --verbose flag as an env var
 ResolvedSlst2020T322286 [tbs][cli] Create proper subcommands and add 'start' subcommand
 ResolvedSlst2020T323681 [tbs.cli] Multiple bugs when running external commands
 ResolvedSlst2020T323709 [tbs.cli] Do not raise an error when an external subcommand fails
 ResolvedSlst2020T323711 [tbs.cli] Options of external subcommands are not forwarded
 ResolvedRaymond_NdibeT323700 [toolforge-cli][build] show fails if the pipelinerun has no 'status'
 ResolvedSlst2020T324824 tbs: user-story 2 - start cli should autogenerate the image name when not passed
 ResolvedSlst2020T324825 tbs: user-story 2 - start cli should give the right error when the git url is not passed
 ResolvedSlst2020T324826 tbs: user-story 2 - start cli should give the right error when the service is down
 ResolveddcaroT331862 [tbs.cli] build start default builder has no tag, and does not pass the admission webhook check
 ResolveddcaroT331873 [tbs.toolsbeta] Push the heroku builder image to the harbor.toolsbeta repository
 ResolveddcaroT331876 [tbs.cli] change the default builder to be just the heroku builder from the harbor repository
 ResolveddcaroT331893 [tsb.cli] don't force a user when running upstream buildpacks
 ResolveddcaroT331897 [webservce] buildservice type is shown as deprecated
 ResolveddcaroT331723 [tbs.buildpack-admission-controller] Restrict the builders to heroku only
 ResolvedRaymond_NdibeT324832 tbs: user-story 4 - show cli should use the latest build if no build id is passed
 ResolveddcaroT324834 tbs: user-story 6 - I want to start a webservice with the image I built earlier.
 ResolvedRaymond_NdibeT293645 [tbs.webservice] Modify the webservice cli to allow the deployment of custom images from harbor
 ResolvedRaymond_NdibeT316327 [tbs] Deploy on tools
 ResolveddcaroT316325 [tbs.buildservice.toolsbeta] Deploy on toolsbeta
 ResolvedRaymond_NdibeT316326 [tbs.buildservice] Ensure we use the deploy.sh standard
 ResolvedRaymond_NdibeT332347 Duplicate the maintain-harbor deployment for "tools"
 ResolvedfnegriT316323 [tbs.harbor.tools] Replicate toolsbeta harbor deployment
 ResolvedNoneT265681 [tbs.tools] Deploy a tool using a buildpack-based image on Toolforge
 ResolveddcaroT316330 [tbs.toolsbeta] Deploy a tool using a buildpack-based image on Toolsbeta
 ResolveddcaroT267616 [tbs.harbor] Puppetize the toolsbeta installation
 ResolvedSlst2020T316232 [tbs.harbor.toolsbeta] Create a postgres DB with trove and plug the current instance there
 ResolvedRaymond_NdibeT293649 [tbs.harbor] Pre-create namespaces
 ResolvedRaymond_NdibeT316339 [tbs.registry-admission.tools] Allow images from harbor on tools
 ResolvedRaymond_NdibeT323689 [webservice] Allow configuring the image registry used to pull
 ResolveddcaroT324566 [webservice] Figure out how to configure depending on the environment it's running
 ResolveddcaroT325141 tbs: user-story 6 - Build and deploy the modified webservice to toolsbeta
 ResolveddcaroT325142 tbs: user-story 6 - Build and deploy the modified webservice to tools

Event Timeline

dcaro triaged this task as High priority.Aug 25 2022, 3:29 PM
dcaro created this task.
dcaro added a project: Toolforge Build Service.
dcaro added a parent task: T194332: [Epic] Make Toolforge a proper platform as a service with push-to-deploy and build packs.
dcaro added a parent task: T267616: [tbs.harbor] Puppetize the toolsbeta installation.Aug 26 2022, 8:25 AM
dcaro removed a parent task: T194332: [Epic] Make Toolforge a proper platform as a service with push-to-deploy and build packs.Aug 26 2022, 8:50 AM
dcaro added a comment.Aug 29 2022, 1:24 PM

Note for the docs, every time you change any configuration for harbor (harbor.yml) you have to run the ./install.sh script.

More notes on the current setup:

  • the configuration is under /srv/ops/harbor
  • the data is under /srv/harbor
  • /srv is already a cinder volume
dcaro assigned this task to Slst2020.Aug 29 2022, 1:25 PM
KHernandez-WMF set the point value for this task to 2.Aug 31 2022, 5:42 PM
KHernandez-WMF moved this task from Backlog to Discussed/Estimated on the Toolforge Build Service board.
KHernandez-WMF moved this task from Discussed/Estimated to Iteration 00 on the Toolforge Build Service board.Aug 31 2022, 5:49 PM
KHernandez-WMF edited projects, added Toolforge Build Service (Iteration 00); removed Toolforge Build Service.
Slst2020 added a comment.Sep 7 2022, 1:25 PM

Document also where it needs to be configured in the README for the buildservice repository:
https://github.com/toolforge/buildservice

@dcaro do you mean how to configure the docker-compose setup to work with the external db?

dcaro added a comment.Sep 7 2022, 2:38 PM
In T316232#8217406, @Slst2020 wrote:

Document also where it needs to be configured in the README for the buildservice repository:
https://github.com/toolforge/buildservice

@dcaro do you mean how to configure the docker-compose setup to work with the external db?

Yep, mostly so it would be easy to automate later or similar.

Slst2020 added a project: User-Slst2020.Sep 12 2022, 6:07 AM
dcaro moved this task from Iteration 00 to Iteration 01 on the Toolforge Build Service board.Sep 27 2022, 8:24 AM
dcaro edited projects, added Toolforge Build Service (Iteration 01); removed Toolforge Build Service (Iteration 00).
bd808 moved this task from Inbox to Doing on the cloud-services-team (Kanban) board.Sep 27 2022, 9:24 PM
Slst2020 moved this task from To Do to Doing on the User-Slst2020 board.Oct 5 2022, 8:31 AM
Slst2020 moved this task from Next Up to In Progress on the Toolforge Build Service (Iteration 01) board.
dcaro updated the task description. (Show Details)Oct 11 2022, 10:26 AM
KHernandez-WMF moved this task from Iteration 01 to Iteration 02 on the Toolforge Build Service board.Oct 11 2022, 4:00 PM
KHernandez-WMF edited projects, added Toolforge Build Service (Iteration 02); removed Toolforge Build Service (Iteration 01).
KHernandez-WMF moved this task from Next Up to In Progress on the Toolforge Build Service (Iteration 02) board.
Slst2020 updated the task description. (Show Details)Oct 12 2022, 5:54 AM
Slst2020 updated the task description. (Show Details)Oct 14 2022, 12:51 PM
Slst2020 updated the task description. (Show Details)Oct 20 2022, 9:06 AM
Slst2020 closed this task as Resolved.Oct 20 2022, 9:41 AM
Slst2020 updated the task description. (Show Details)
dcaro moved this task from In Progress to Done on the Toolforge Build Service (Iteration 02) board.Oct 25 2022, 10:07 AM
fnegri mentioned this in T332208: [tbs] Harbor broken in toolsbeta because of missing db password.Mar 15 2023, 6:33 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL