Page MenuHomePhabricator

Nightly builds should be a shallow git clone
Closed, ResolvedPublic

Description

The nightly builds appear to include a complete git clone. There have been complaints about the time it takes to unzip the file. It is also unfriendly to people not sitting on a high speed data unlimited connection.


Version: core-(2.0)
Severity: enhancement

Details

Reference
bz72898

Related Objects

Event Timeline

bzimport raised the priority of this task from to High.Nov 22 2014, 3:59 AM
bzimport set Reference to bz72898.
jayvdb created this task.Nov 3 2014, 12:37 PM

I set up the nightly dumps and I'll fix it :)

I made some improvements, Is it still valid?

Even with shallow clone loads of garbage (GARBAGE! I'm angry) were being downloaded (for instance 26M for compat, 14M for i18n).

At first I changed git pull to removing it completely and re-cloning it again. some improvements were made but not much.
I used git gc to reduce git objects but it didn't work out very well, I used more aggressive methods (like --prune=all and another git gc for i18n submodule) and now core is less than 10M and compat is 20M. The only thing that worries me it's that --prune=all maybe causes breakage of pwb.py version (or version.py in compat). I will check and tell you soon.

I also fixed broken links

About showing file sizes since it's plain html, It's not possible (rewriting that page in php won't be hard but another bug would be a better place for it)

http://tools.wmflabs.org/pywikibot/core.tar.gz - 8.5M - very nice!! As mentioned, need to test version functionality.

Output:
amir@amir-Inspiron-N5110:~/Downloads/core$ python pwb.py version
Pywikibot: [https] r-pywikibot-core.git (47ba777, g3, 2014/11/10, 00:50:49, ok)
Release version: 2.0b2
Even though the the g number is nonsense but the date and first number is completely okay, output of git log is also okay.

and -verbose ?

version.py is the same with or without verbose (obviously):

A simple script with verbose looks okay to me:
http://paste.ubuntu.com/8933845/