Page MenuHomePhabricator

Refactor Gulpfile.js tasks into separate files
Closed, ResolvedPublic

Description

Currently all of Gulp tasks that produce the final www.wikipedia.org page are located in a single Gulpfile.js file in the root of the portals repo. This makes the build process difficult to understand and maintain. Splitting the Gulp tasks into separate files would help contributors understand what each function is responsible for, and eventually, may make it easier to refactor and replace part of the build process in the future.

Splitting a Gulpfile is also recommended in the Gulp.js documentation:
https://gulpjs.com/docs/en/getting-started/javascript-and-gulpfiles#splitting-a-gulpfile

All Gulp.js related code should be placed in folder called "gulpfile.js"

The first piece of work required for this refactor is to create a folder called /gulpfile.js/ at the root of the repo, and move the /gulpfile.js file into /gulpfile.js/index.js as is recommended by the Gulp.js documentation. https://gulpjs.com/docs/en/getting-started/javascript-and-gulpfiles#splitting-a-gulpfile

Gulp task functions should be split into separate files

Currently most Gulp tasks are defined as functions, and the right below the function, a gulp.task() call registers the function as a Gulp task. This task involves moving the functions into separate files, but keeping the gulp.task definitions in /gulpfile.js/index.js (changing export names, as recommended by the Gulp.js documentation, can be done in a followup task).

Example of a current Gulp task in gulpfile.js:

gulpfile.js
 function compileHandlebars() {
	requirePortalParam();
	return gulp.src( getConfig().hb.src )
		.pipe( plugins.compileHandlebars( getConfig().hb.templateData, getConfig().hb.options ) )
		.pipe( plugins.rename( 'index.html' ) )
		.pipe( gulp.dest( getBaseDir() ) );
}
gulp.task( 'compile-handlebars', compileHandlebars );

Becomes a separate file called gulpfile.js/handlebars.js with the necessary functions and a module export

gulpfile.js/handlebars.js

 function compileHandlebars() {
	requirePortalParam();
	return gulp.src( getConfig().hb.src )
		.pipe( plugins.compileHandlebars( getConfig().hb.templateData, getConfig().hb.options ) )
		.pipe( plugins.rename( 'index.html' ) )
		.pipe( gulp.dest( getBaseDir() ) );
}
exports.compileHandlebars = compileHandlebars;

And then in gulpfile.js/index.js:

gulpfile.js/index.js

const { compileHandlebars } = require('./handelbars.js'); 

gulp.task( 'compile-handlebars', compileHandlebars );

NOTES:

  • This work should be split across multiple patches. Each new file created can be a separate patch.
  • Most tasks share common dependancy functions like requirePortalParam() and getConfig(). These will have to be separated first, and then imported by each individual file that requires their usage.
  • Many tasks require plugins that are loaded via plugins = gulpLoadPlugins() . This dependancy should be carried over into the separate files as well.
  • The changes can be tested by running npm run build-all-portals and viewing the contents of the prod folder. That task should continue to run successfully.

New file structure

A folder called /gulpfile.js should be created at the root of the repo and the existing gulp file should be moved to /gulpfile.js/index.js. The functions inside index.js can be split into different files in the following way:

New files to be created in the /gulpfile.js/ folder

help.js
The first task, help() can be moved into its own file.
https://gerrit.wikimedia.org/r/plugins/gitiles/wikimedia/portals/+/refs/heads/master/gulpfile.js#23,

config.js
https://gerrit.wikimedia.org/r/plugins/gitiles/wikimedia/portals/+/refs/heads/master/gulpfile.js#52

  • requirePortalParam()
  • getBaseDir()
  • getProdDir()
  • getConfig

handlebars.js
https://gerrit.wikimedia.org/r/plugins/gitiles/wikimedia/portals/+/refs/heads/master/gulpfile.js#146

  • compileHandlebars()

postcss.js
https://gerrit.wikimedia.org/r/plugins/gitiles/wikimedia/portals/+/refs/heads/master/gulpfile.js#163

  • postCSS()
  • validatePostCSS()
  • lintCSS()

prod.js

  • inlineAssets()
  • cleanProdJS()
  • copyTranslationFiles()
  • concatMinifyJS()
  • minifyHTML()
  • copyImages()
  • createProdSymlink()

javascript.js

  • lintJS()

stats.js

  • updateStats()

meta.js

  • fetchMeta()

sprites.js

  • cleanSprites()
  • createSvgSprite()

scap-urls.js

  • updateURLsToPurge()

dev.js

  • watch()

Note that these refactors can be split into separate patches. After this clean-up, the file in gulpfile.js/index.js should just contain imports and gulp.task definitions.

Event Timeline

Hey, @Tsiruot i want to work on this task as i observed. you claimed three of the task @Jdrewniak without submitting the patch for it .....Which i think is not a good practice. every one want to contribute..
Thanks : )

@Bharatkhatri351 , @Tsiruot , this is a fairly large task, and the description mentions it should be split into multiple patches. So since it would be a good learning experience for both of you, why don't we split the work in half. I assume @Tsiruot is working on the first patch, which is creating the folder and new index.js file, but after that is merged, we can split the creation of the remaining files down the middle (there are more than ten files needing creation here after all).

@Bharatkhatri351 I am sorry if it felt like it was unfair to you but I have already submitted two patches for other tasks and now I have also started working on this task. Like @Jdrewniak said this task can be divided into multiple patches, so let's work together on this task and get it done :)

@Bharatkhatri351 I am sorry if it felt unfair to you. I have already submitted two patches for other tasks and now I have also started working on this task. Like @Jdrewniak said this task can be divided into multiple patches, so let's work together on this task and get it done :)

Thats seem cool....Yaaa can work on it together :)

@Bharatkhatri351 We could discuss about it on Zulip

ok good idea...

Change 672210 had a related patch set uploaded (by Ishan Saini; owner: Ishan Saini):
[wikimedia/portals@master] Move code from gulpfile.js to gulpfile.js/index.js

https://gerrit.wikimedia.org/r/672210

@Jdrewniak I have moved the code of gulpfile.js into gulpfile.js/index.js. Kindly review the changes and merge them so that @Bharatkhatri351 can proceed further.

@Bharatkhatri351 You can go ahead and create help.js file in your next patch once the changes are merged.

Change 672210 merged by jenkins-bot:
[wikimedia/portals@master] Move code from gulpfile.js to gulpfile.js/index.js

https://gerrit.wikimedia.org/r/672210

hey,
@Jdrewniak
I have moved "helptask" to help.js..kindly review the changes.

@Tsiruot
now u can work on further tasks..after merge.. : )

Change 672515 had a related patch set uploaded (by Bharatkhatri; owner: Bharatkhatri):
[wikimedia/portals@master] Added help.js file inside gulpfile.js folder

https://gerrit.wikimedia.org/r/672515

Change 672515 had a related patch set uploaded (by Bharatkhatri; owner: Bharatkhatri):
[wikimedia/portals@master] Added help.js file inside gulpfile.js folder

https://gerrit.wikimedia.org/r/672515

Change 672515 merged by jenkins-bot:
[wikimedia/portals@master] Added help.js file inside gulpfile.js folder

https://gerrit.wikimedia.org/r/672515

Change 673772 had a related patch set uploaded (by Ishan Saini; owner: Ishan Saini):
[wikimedia/portals@master] Adds config.js file in gulpfile.js directory

https://gerrit.wikimedia.org/r/673772

Change 673772 merged by jenkins-bot:
[wikimedia/portals@master] Adds config.js file in gulpfile.js directory

https://gerrit.wikimedia.org/r/673772

Change 673779 had a related patch set uploaded (by Bharatkhatri; owner: Bharatkhatri):
[wikimedia/portals@master] Adds config.js file in gulpfile.js directory

https://gerrit.wikimedia.org/r/673779

Change 673779 merged by jenkins-bot:
[wikimedia/portals@master] Adds handlebar.js file in gulpfile.js directory

https://gerrit.wikimedia.org/r/673779

Change 674580 had a related patch set uploaded (by Ishan Saini; owner: Ishan Saini):
[wikimedia/portals@master] Adds postcss.js file in gulpfile.js directory

https://gerrit.wikimedia.org/r/674580

Change 674580 merged by jenkins-bot:
[wikimedia/portals@master] Adds postcss.js file in gulpfile.js directory

https://gerrit.wikimedia.org/r/674580

Change 674630 had a related patch set uploaded (by Bharatkhatri; owner: Bharatkhatri):
[wikimedia/portals@master] Added prod.js file inside gulpfile.js directory

https://gerrit.wikimedia.org/r/674630

Change 674630 merged by jenkins-bot:
[wikimedia/portals@master] Added prod.js file inside gulpfile.js directory

https://gerrit.wikimedia.org/r/674630

Change 675646 had a related patch set uploaded (by Ishan Saini; author: Ishan Saini):
[wikimedia/portals@master] Adds javascript.js file in gulpfile.js directory

https://gerrit.wikimedia.org/r/675646

I have created a pull request and created files javascript.js, meta.js, scap-urls.js. Can someone please review my changes. Here is the link.

If there is any problem, I am happy to fix it.

Hi @Helix17, could you please edit the commit message to follow https://www.mediawiki.org/wiki/Gerrit/Commit_message_guidelines ? Thanks in advance! :)

Change 676715 had a related patch set uploaded (by Priyaraj17; author: Priyaraj17):

[wikimedia/portals@master] Gulp.js tasks:

https://gerrit.wikimedia.org/r/676715

Hi @Helix17, could you please edit the commit message to follow https://www.mediawiki.org/wiki/Gerrit/Commit_message_guidelines ? Thanks in advance! :)

I changed the commit message. Thanks

Change 676715 had a related patch set uploaded (by Aklapper; author: Priyaraj17):

[wikimedia/portals@master] Refactor Gulpfile.js tasks into separate files

https://gerrit.wikimedia.org/r/676715

Change 675646 merged by jenkins-bot:

[wikimedia/portals@master] Adds javascript.js file in gulpfile.js directory

https://gerrit.wikimedia.org/r/675646

Change 680261 had a related patch set uploaded (by Bharatkhatri; author: Bharatkhatri):

[wikimedia/portals@master] Added stat.js in gulpfile.js folder

https://gerrit.wikimedia.org/r/680261

Change 680261 merged by jenkins-bot:

[wikimedia/portals@master] Added stats.js in gulpfile.js folder

https://gerrit.wikimedia.org/r/680261

Change 681719 had a related patch set uploaded (by Ishan Saini; author: Ishan Saini):

[wikimedia/portals@master] Add meta.js file in gulpfile.js directory

https://gerrit.wikimedia.org/r/681719

Change 681719 merged by jenkins-bot:

[wikimedia/portals@master] Add meta.js file in gulpfile.js directory

https://gerrit.wikimedia.org/r/681719

Change 683384 had a related patch set uploaded (by Bharatkhatri; author: Bharatkhatri):

[wikimedia/portals@master] Added sprites.js in gulpfile.js directory

https://gerrit.wikimedia.org/r/683384

Change 683384 merged by jenkins-bot:

[wikimedia/portals@master] Added sprites.js in gulpfile.js directory

https://gerrit.wikimedia.org/r/683384

Change 691590 had a related patch set uploaded (by Ishan Saini; author: Ishan Saini):

[wikimedia/portals@master] Add scap-urls.js file in gulpfile.js directory

https://gerrit.wikimedia.org/r/691590

Change 691590 merged by jenkins-bot:

[wikimedia/portals@master] Add scap-urls.js file in gulpfile.js directory

https://gerrit.wikimedia.org/r/691590

Change 693141 had a related patch set uploaded (by Bharatkhatri; author: Bharatkhatri):

[wikimedia/portals@master] Add dev.js file in gulpfile.js folder

https://gerrit.wikimedia.org/r/693141

Change 693141 merged by jenkins-bot:

[wikimedia/portals@master] Add dev.js file in gulpfile.js folder

https://gerrit.wikimedia.org/r/693141