Page MenuHomePhabricator

Simplify WDQS Packaging
Closed, ResolvedPublic

Description

Background information

Currently, the WDQS build pipeline is unable to be built using standard Java protocols due to the WDQS GUI git submodule located in the same directory.

What

We propose simplifying the build process by separating the projects and building them individually

How

Right now we have a couple of options
The best one seems to be migrating the WDQS GUI to the node service template and service runner. This would allow both projects to be released independently.

We could also do some kind of static file serve for the WDQS GUI and keep the version number in the WDQS project.

Open questions

Please add any open questions!

Acceptance criteria

  • Git submodule has been removed from WDQS
  • WDQS GUI can be deployed successfully
  • WDQS can be built and deployed using typical Maven processes

Event Timeline

Mstyles created this task.Dec 21 2019, 12:52 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptDec 21 2019, 12:52 AM
Gehel added a comment.Jan 7 2020, 11:09 AM

I'm completely unfamiliar with the GUI project itself. But my understanding is that it uses nodejs only as a build tool and that the resulting artifact is a purely static HTML+JS application. service-runner is meant as a way to run nodejs applications (again, I have a limited understanding). So I think that we don't need service-runner, just a way to deploy static HTML.

I'm completely unfamiliar with the GUI project itself. But my understanding is that it uses nodejs only as a build tool and that the resulting artifact is a purely static HTML+JS application.

Yup

service-runner is meant as a way to run nodejs applications (again, I have a limited understanding). So I think that we don't need service-runner, just a way to deploy static HTML.

indeed we don't need service runner, however in the longer term i highly doubt that this thing will remain a static html based application :)

I'm completely unfamiliar with the GUI project itself. But my understanding is that it uses nodejs only as a build tool and that the resulting artifact is a purely static HTML+JS application. service-runner is meant as a way to run nodejs applications (again, I have a limited understanding). So I think that we don't need service-runner, just a way to deploy static HTML.

All that is correct, but to add one more constraint, our pipeline isn't tuned to creating images that serve static HTML yet, so I 've proposed to shoehorn this for now using service-runner.

@akosiaris Could we possibly use miscweb in front of a VM as an interim to serve up the static files before moving to service template?

Had a quick sync meeting with WMDE. The outcome of that was to use this node patch as a starting point for service runner. It's unclear whether or not blubber needs to be involved in this process. Also, ideally the public image for the WDQS UI would be eliminated in favor of the new image used for this new build process.

@akosiaris Could we possibly use miscweb in front of a VM as an interim to serve up the static files before moving to service template?

Just to make sure I have fully understood, what's miscweb in this context? Also, why interim ? I get a sense of urgency from this, is my feeling correct? Or have I misunderstood something?

Had a quick sync meeting with WMDE. The outcome of that was to use this node patch as a starting point for service runner.

Ah cool, so that's to be resurrected. Nice!

It's unclear whether or not blubber needs to be involved in this process.

It is to be involved. It's a critical part of the pipeline that generates the image.

I made the patch and would love to continue working on it if it's possible to deploy and serve traffic using that. If understood the long-time-ago discussions correctly, currently it's not possible to separate GUI and backend. Correct me if that situation has changed.

After a bunch of discussion with the team, it's been decided that removing the gui submodule from the RDF repository will suffice for now. That will fix our broken build issues (see https://phabricator.wikimedia.org/T242640) @Ladsgroup I definitely think you should work on that patch and getting things going with service runner if you have the bandwidth.

Change 565414 had a related patch set uploaded (by Mstyles; owner: Mstyles):
[wikidata/query/rdf@master] [gui] remove gui submodule and references

https://gerrit.wikimedia.org/r/565414

Change 565414 merged by jenkins-bot:
[wikidata/query/rdf@master] [gui] remove gui submodule and references

https://gerrit.wikimedia.org/r/565414

Gehel added a comment.Jan 17 2020, 1:17 PM

I made the patch and would love to continue working on it if it's possible to deploy and serve traffic using that. If understood the long-time-ago discussions correctly, currently it's not possible to separate GUI and backend. Correct me if that situation has changed.

It depends what you mean by "possible". It would require some work, but there is no reason that it couldn't be done. It seems to me that all the dependencies we have between GUI and backend are accidental.

I made the patch and would love to continue working on it if it's possible to deploy and serve traffic using that. If understood the long-time-ago discussions correctly, currently it's not possible to separate GUI and backend. Correct me if that situation has changed.

It depends what you mean by "possible". It would require some work, but there is no reason that it couldn't be done. It seems to me that all the dependencies we have between GUI and backend are accidental.

I personally think we should decouple it but I don't know if there's anything I can do to move it forward.

dependencies we have between GUI and backend are accidental

There are not really any dependencies at all.
Other than the fact that they are currently deployed together.

After a bunch of discussion with the team, it's been decided that removing the gui submodule from the RDF repository will suffice for now. That will fix our broken build issues (see https://phabricator.wikimedia.org/T242640) @Ladsgroup I definitely think you should work on that patch and getting things going with service runner if you have the bandwidth.

Does this mean this ticket essentially will not be worked on any further by you?

@Addshore that's correct, after removing the gui submodule, I won't be doing any further work

Change 566370 had a related patch set uploaded (by Mstyles; owner: Mstyles):
[wikidata/query/rdf@master] [gui] remove duplicate profile

https://gerrit.wikimedia.org/r/566370

Change 566370 merged by jenkins-bot:
[wikidata/query/rdf@master] [gui] remove duplicate profile

https://gerrit.wikimedia.org/r/566370

There also appears to be a pom file in the gui repo.
If you feel that is not needed (we are not sure from our side but i guess it is not) it would great to also have that, and any other left over java things removed from that repo!

In terms of eventually moving this to the deployment pipeline etc that will thus continue to be tracked in T192006 and T210286

Change 566571 had a related patch set uploaded (by Mstyles; owner: Mstyles):
[wikidata/query/gui@master] Remove pom from gui

https://gerrit.wikimedia.org/r/566571

Change 566571 merged by jenkins-bot:
[wikidata/query/gui@master] Remove pom from gui

https://gerrit.wikimedia.org/r/566571

Anything I can do to help with this? It's a blocker for us in Release Engineering (through T210286 and T239981 to T236576).

@Jdforrester-WMF WMDE will be taking on responsibility for any new deployment methods. That work will be tracked in T192006 and T210286.

TJones closed this task as Resolved.Feb 12 2020, 4:36 PM