Page MenuHomePhabricator

[Epic,builds-api,orchestrator,webservice,jobs-api] Make Toolforge a proper platform as a service with push-to-deploy and build packs
Open, MediumPublic

Description

Make usage of Toolforge more familiar to users of commercial Platform as a Service (PaaS) products (Heroku, DigitalOcean, Google App Engine, etc) by implementing a Container based PaaS product that provides "build pack" and "push to deploy" workflows for deploying and scaling Tool owned processes.

  • roughly estimated 9-12 month project
  • initial: 1 root, 1 SWE, .25 tech writer, .25 community liaison
  • sustaining: .5 root, .5 SWE, .10 tech writer, .10 community liaison

https://wikitech.wikimedia.org/wiki/Wikimedia_Cloud_Services_team/EnhancementProposals/Toolforge_push_to_deploy

Design Documentation: https://wikitech.wikimedia.org/wiki/Wikimedia_Cloud_Services_team/EnhancementProposals/Toolforge_Buildpack_Implementation

Related Objects

View Standalone Graph
This task is connected to more than 200 other tasks. Only direct parents and subtasks are shown here. Use View Standalone Graph to show more of the graph.
StatusSubtypeAssignedTask
ResolvedLucasWerkmeister
Resolvedmatmarex
ResolvedLegoktm
ResolvedLegoktm
Opendcaro
DeclinedNone
DeclinedNone
DeclinedBstorm
Resolveddcaro
ResolvedBstorm
ResolvedLegoktm
ResolvedLegoktm
Resolveddcaro
ResolvedLegoktm
DuplicateNone
Resolveddcaro
Resolveddcaro
Resolveddcaro
ResolvedSlst2020
Resolveddcaro
Resolveddcaro
Resolveddcaro
Resolveddcaro
Resolveddcaro
StalledNone
ResolvedRaymond_Ndibe
OpenNone
Resolveddcaro
Duplicatedcaro
Resolveddcaro
Resolveddcaro
StalledNone
ResolvedSlst2020
Resolvedaborrero
ResolvedSecuritydcaro

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

Does the push-to-deploy pipeline accept early adopters? I’d gladly volunteer as a guinea pig.

@Sascha That would be wonderful to get feedback! What tool / language are you hoping to try out?

@nskaggs, it’s in Go, although I’m working on a web frontend that’ll eventually have a part written in JavaScript/React. Here’s my current “release process” that I’m hoping to make less manual. The environment variable GOOS=linux can be omitted when the compiler runs on a Linux machine.

git clone https://github.com/brawer/wikidata-qrank && cd wikidata-qrank
go test ./cmd/...
for tool in `ls cmd` ; do GOOS=linux go build -ldflags="-s -w" ./cmd/$tool && scp ./$tool sascha@bastion.toolforge.org:/data/project/qrank/bin/$tool ; done

Hi @Sascha, there's no timeline yet, but I'll ping you whenever the pipeline is ready to try, I'll really appreciate your input :)

We will support three main builders, paketo that supports building golang apps (you code might need some tweaking for this one), google cloud, that also supports golang, and our own toolforge one that does not support golang yet, though might in the future.

If you have any other needs please share and we'll try to accommodate for them too.

dcaro changed the status of subtask T304532: buildservice: migrate to helmfile from of kustomize from Open to In Progress.
dcaro renamed this task from [Epic] Make Toolforge a proper platform as a service with push-to-deploy and build packs to [Epic,builds-api,orchestrator,webservice,jobs-api] Make Toolforge a proper platform as a service with push-to-deploy and build packs.Tue, Mar 5, 3:55 PM