Page MenuHomePhabricator
Create Task
Maniphest T178554

Create a Tool on ToolForge for automating www portal builds
Closed, DeclinedPublic
Actions

Description

In order to create the production HTML/CSS/JS for the www portals, a node-based build step (using the Gulp build tool) is run that:

  • pull new statistics
  • compiles js/css/image assets
  • updates translations
  • processes templates
  • minifies/combines/compiles all the above into static HTML/CSS/JS

Currently this build step is run on a developer machine and the output gets submitted as a patch to Gerrit. This build step could be automated if it was periodically run on ToolForge by a Tool instead of manually on a developer machine.

To automate this build step on ToolForge, a new Tool needs to be created that does the following:

  • Pulls the repo from Gerrit
  • Runs the build-step outlined above
  • Submits a new patch to Gerrit with the updated build

Apart from being able to successfully run the build and create the commit (with the Gerrit hooks), this tool would require a Gerrit account in order to submit the patch. Submitting the patch to Gerrit for manual review is still preferable in case the tool messes something up. The patch would then be manually +2d by a developer as it is now.

Related Objects
Search...

Event Timeline

Jdrewniak created this task.Oct 19 2017, 9:15 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 19 2017, 9:15 AM
Jdrewniak added a parent task: T140159: [epic] Wikipedia.org Portal: automating updates for the portal.Oct 19 2017, 9:16 AM
Jdrewniak added a parent task: T177345: [epic] Wikipedia.org portal: creating and pushing updates.Oct 19 2017, 9:19 AM
Jdrewniak added a subtask: T177348: wikipedia.org automation: gerrit push and CI.Oct 19 2017, 9:23 AM
Jdrewniak added a subtask: T177347: wikipedia.org automation: portal translations automation.
Jdrewniak added a subtask: T177346: wikipedia.org automation: portal stats automation.
Jdrewniak added a comment.Oct 19 2017, 9:34 AM

So far I've created a node tool on toolforge that is able to pull the repo, run the build, and create a commit (it can create a gerrit commit if it has credentials).

https://phabricator.wikimedia.org/source/tool-www-portal-builder/

The tool also has a web front-end that lets users compare the new build created by the tool with the version currently in production.

https://tools.wmflabs.org/www-portal-builder/

Jdrewniak added a subscriber: Joe.Oct 19 2017, 9:52 AM
debt triaged this task as Medium priority.Oct 19 2017, 4:34 PM
debt added a subscriber: greg.

I'll set up a meeting to go over the concept and details of this ticket and functionality.

Krinkle awarded a token.Oct 20 2017, 3:43 AM
zhuyifei1999 subscribed.Oct 20 2017, 6:59 PM
debt closed this task as Resolved.Oct 26 2017, 3:14 PM
debt moved this task from Backlog to Done on the Discovery-Portal-Sprint board.

Based on a meeting we had with RelEng (T177350), we've decided not to do the following steps:

To automate this build step on ToolForge, a new Tool needs to be created that does the following:
Pulls the repo from Gerrit
Runs the build-step outlined above
Submits a new patch to Gerrit with the updated build

Closing this ticket as we're done with what we need to do, based on the description.

Quiddity created subtask T179223: Cleanup notes for tool-www-portal-builder.Oct 28 2017, 1:24 AM
Jdrewniak added a comment.Nov 3 2017, 2:56 PM

Following up on @debt 's comments. Instead of running the portal build on toolForge, we decided it would be better to run the build in CI with Jenkins, since that wouldn't require maintaining a tool, and CI already has Gerrit permissions (and Jenkin's is kind of designed for building deployment artifacts).

This task is being replaced with T179694

debt closed subtask T177347: wikipedia.org automation: portal translations automation as Resolved.Nov 28 2017, 4:46 PM
debt closed subtask T177346: wikipedia.org automation: portal stats automation as Resolved.
debt closed subtask T177348: wikipedia.org automation: gerrit push and CI as Resolved.
debt closed subtask T179223: Cleanup notes for tool-www-portal-builder as Declined.Dec 5 2017, 4:12 PM
Legoktm changed the task status from Resolved to Declined.Dec 5 2017, 7:12 PM
Legoktm subscribed.

Setting status to declined, because this didn't happen.

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL