Page MenuHomePhabricator
Create Task
Maniphest T200600

Pwb nightly downloads and /shared/pywikibot/core are outdated
Closed, ResolvedPublic
Actions

Description

framawiki@tools-bastion-02:/shared/pywikibot/core$ git log
commit 266c42ed34c91651b767e11d6522f6da62ec2ef1
Author: jenkins-bot <jenkins-bot@gerrit.wikimedia.org>
Date:   Fri Jul 13 16:48:42 2018 +0000

    Merge "[cleanup] Code cleanup"
framawiki@tools-bastion-02:/data/project/pywikibot/public_html/core$ git log
commit 266c42ed34c91651b767e11d6522f6da62ec2ef1
Author: jenkins-bot <jenkins-bot@gerrit.wikimedia.org>
Date:   Fri Jul 13 16:48:42 2018 +0000

    Merge "[cleanup] Code cleanup"

framawiki@tools-bastion-02:/data/project/pywikibot/public_html/core$ python pwb.py version
Pywikibot: [https] r-pywikibot-core.git (266c42e, g1, 2018/07/13, 16:48:42, OUTDATED)
Release version: 3.1.dev0
requests version: 2.2.1
  cacerts: /etc/ssl/certs/ca-certificates.crt
    certificate test: ok
Python: 2.7.6 (default, Nov 23 2017, 15:49:48) 
[GCC 4.8.4]
Toolforge hostname: tools-bastion-02
PYWIKIBOT2_DIR: Not set
PYWIKIBOT2_DIR_PWB: 
PYWIKIBOT2_NO_USER_CONFIG: 2
Config base dir: /mnt/nfs/labstore-secondary-tools-project/pywikibot/public_html/core

And finally by downloading and unzipping the nightly zip (http://tools.wmflabs.org/pywikibot/core.zip):

framawiki@tools-bastion-02:~/core$ python pwb.py version
Pywikibot: [https] r-pywikibot-core.git (266c42e, g1, 2018/07/13, 16:48:42, OUTDATED)
Release version: 3.1.dev0
requests version: 2.2.1
  cacerts: /etc/ssl/certs/ca-certificates.crt
    certificate test: ok
Python: 2.7.6 (default, Nov 23 2017, 15:49:48) 
[GCC 4.8.4]
Toolforge hostname: tools-bastion-02
PYWIKIBOT2_DIR: Not set
PYWIKIBOT2_DIR_PWB: 
PYWIKIBOT2_NO_USER_CONFIG: 2
Config base dir: /mnt/nfs/labstore-secondary-tools-home/framawiki/core

It looks like all of them are not updated since 2018/07/13.
It's very problematic since lot (praticly all?) tools bots are using these patchs to use the last up-to-date version of the library.

Related Objects

Event Timeline

Framawiki triaged this task as High priority.Jul 28 2018, 10:47 AM
Framawiki created this task.
Restricted Application added subscribers: pywikibot-bugs-list, Aklapper. · View Herald TranscriptJul 28 2018, 10:47 AM
Framawiki added subscribers: Legoktm, valhallasw, Ladsgroup.Jul 28 2018, 10:52 AM
Dvorapa added a subscriber: Dvorapa.Jul 28 2018, 11:30 AM
Legoktm added a comment.Jul 28 2018, 4:12 PM

Bunch of stuff like:

rm: cannot remove ‘public_html.old/core/pywikibot/userinterfaces/__pycache__/transliteration.cpython-34.pyc’: Permission denied

OK so:

-rw-r--r-- 1 framawiki wikidev 73264 Jul 13 15:36 transliteration.cpython-34.pyc

???

I'm unable to remove it...did you maybe run the script under your own account instead of using the pwb tool?

zhuyifei1999 added a subscriber: zhuyifei1999.EditedJul 28 2018, 5:14 PM

FWIW:

tools.pywikibot@tools-bastion-02:~$ stat public_html.old/core/pywikibot/userinterfaces/__pycache__/
  File: ‘public_html.old/core/pywikibot/userinterfaces/__pycache__/’
  Size: 4096      	Blocks: 8          IO Block: 1048576 directory
Device: 24h/36d	Inode: 122168003   Links: 2
Access: (0755/drwxr-xr-x)  Uid: (14889/framawiki)   Gid: (  500/ wikidev)
Access: 2018-07-13 15:36:16.778751917 +0000
Modify: 2018-07-13 15:36:16.902752090 +0000
Change: 2018-07-13 15:36:16.902752090 +0000
 Birth: -
tools.pywikibot@tools-bastion-02:~$ take public_html.old/core/pywikibot/userinterfaces/__pycache__/
public_html.old/core/pywikibot/userinterfaces/__pycache__/: You need to share a group with the file

Why are these directories (I mean public_html.old/core/pywikibot/userinterfaces) even group-writable?

zhuyifei1999 added a comment.Jul 28 2018, 5:49 PM

@Legoktm https://github.com/pywikibot/Pywikibot-nightly-creator/pull/5 sets umask 0022.

We perhaps also should set umask in .profile. I can recursively chown and/or chmod afterwards.

Legoktm added a comment.Jul 28 2018, 9:11 PM
In T200600#4458939, @zhuyifei1999 wrote:

@Legoktm https://github.com/pywikibot/Pywikibot-nightly-creator/pull/5 sets umask 0022.

Merged, thank you.

We perhaps also should set umask in .profile. I can recursively chown and/or chmod afterwards.

Seems reasonable to me. Could you chown the existing stuff so that the script can run?

Stashbot added a subscriber: Stashbot.Jul 29 2018, 4:13 AM

Mentioned in SAL (#wikimedia-cloud) [2018-07-29T04:13:52Z] <zhuyifei1999_> recursively chown /data/project/pywikibot/public_html{,.old}/core/ to tools.pywikibot T200600

zhuyifei1999 added a comment.Jul 29 2018, 4:17 AM
tools.pywikibot@tools-bastion-05:~$ find . \! -user tools.pywikibot | xargs ls -ld
-rw-r--r-- 1 root       tools.pywikibot     988 Mar 18  2014 ./...DATA.crontab
-rw-r--r-- 1 ladsgroup  tools.pywikibot    1008 Mar 28  2014 ./gitlog.py
-rw-r--r-- 1 valhallasw tools.pywikibot  152658 May  1  2014 ./public_html.20160522/125575_new.txt
-rw-r--r-- 1 valhallasw tools.pywikibot  155682 May  1  2014 ./public_html.20160522/125575_old.txt
-rw-r--r-- 1 valhallasw tools.pywikibot      92 Nov 20  2014 ./public_html.20160522/tests/upload.php
-rw-r--r-- 1 framawiki  wikidev             963 Jul 23 15:44 ./public_html/core/mwparserfromhell/compat.pyc
-rw-r--r-- 1 framawiki  wikidev            3516 Jul 23 15:44 ./public_html/core/mwparserfromhell/definitions.pyc
-rw-r--r-- 1 framawiki  wikidev             929 Jul 23 15:44 ./public_html/core/mwparserfromhell/__init__.pyc
-rw-r--r-- 1 framawiki  wikidev            3696 Jul 23 15:44 ./public_html/core/mwparserfromhell/nodes/argument.pyc
-rw-r--r-- 1 framawiki  wikidev            1783 Jul 23 15:44 ./public_html/core/mwparserfromhell/nodes/comment.pyc
-rw-r--r-- 1 framawiki  wikidev            4182 Jul 23 15:44 ./public_html/core/mwparserfromhell/nodes/external_link.pyc
-rw-r--r-- 1 framawiki  wikidev            7047 Jul 23 15:44 ./public_html/core/mwparserfromhell/nodes/extras/attribute.pyc
-rw-r--r-- 1 framawiki  wikidev             489 Jul 23 15:44 ./public_html/core/mwparserfromhell/nodes/extras/__init__.pyc
-rw-r--r-- 1 framawiki  wikidev            3905 Jul 23 15:44 ./public_html/core/mwparserfromhell/nodes/extras/parameter.pyc
-rw-r--r-- 1 framawiki  wikidev            3232 Jul 23 15:44 ./public_html/core/mwparserfromhell/nodes/heading.pyc
-rw-r--r-- 1 framawiki  wikidev            6726 Jul 23 15:44 ./public_html/core/mwparserfromhell/nodes/html_entity.pyc
-rw-r--r-- 1 framawiki  wikidev            3416 Jul 23 15:44 ./public_html/core/mwparserfromhell/nodes/__init__.pyc
-rw-r--r-- 1 framawiki  wikidev           13303 Jul 23 15:44 ./public_html/core/mwparserfromhell/nodes/tag.pyc
-rw-r--r-- 1 framawiki  wikidev           14943 Jul 23 15:44 ./public_html/core/mwparserfromhell/nodes/template.pyc
-rw-r--r-- 1 framawiki  wikidev            2226 Jul 23 15:44 ./public_html/core/mwparserfromhell/nodes/text.pyc
-rw-r--r-- 1 framawiki  wikidev            3421 Jul 23 15:44 ./public_html/core/mwparserfromhell/nodes/wikilink.pyc
-rw-r--r-- 1 framawiki  wikidev           12035 Jul 23 15:44 ./public_html/core/mwparserfromhell/parser/builder.pyc
-rw-r--r-- 1 framawiki  wikidev            4386 Jul 23 15:44 ./public_html/core/mwparserfromhell/parser/contexts.pyc
-rw-r--r-- 1 framawiki  wikidev            4041 Jul 23 15:44 ./public_html/core/mwparserfromhell/parser/__init__.pyc
-rw-r--r-- 1 framawiki  wikidev           52937 Jul 23 15:44 ./public_html/core/mwparserfromhell/parser/tokenizer.pyc
-rw-r--r-- 1 framawiki  wikidev            4703 Jul 23 15:44 ./public_html/core/mwparserfromhell/parser/tokens.pyc
-rw-r--r-- 1 framawiki  wikidev           22212 Jul 23 15:44 ./public_html/core/mwparserfromhell/smart_list.pyc
-rw-r--r-- 1 framawiki  wikidev            6725 Jul 23 15:44 ./public_html/core/mwparserfromhell/string_mixin.pyc
-rw-r--r-- 1 framawiki  wikidev            2411 Jul 23 15:44 ./public_html/core/mwparserfromhell/utils.pyc
-rw-r--r-- 1 framawiki  wikidev           32971 Jul 23 15:44 ./public_html/core/mwparserfromhell/wikicode.pyc
-rw-r--r-- 1 framawiki  wikidev           16999 Jul 23 15:44 ./public_html/core/pywikibot/bot_choice.pyc
-rw-r--r-- 1 framawiki  wikidev           80788 Jul 23 15:44 ./public_html/core/pywikibot/bot.pyc
-rw-r--r-- 1 framawiki  wikidev           16726 Jul 23 15:44 ./public_html/core/pywikibot/comms/http.pyc
-rw-r--r-- 1 framawiki  wikidev             226 Jul 23 15:44 ./public_html/core/pywikibot/comms/__init__.pyc
-rw-r--r-- 1 framawiki  wikidev            7584 Jul 23 15:44 ./public_html/core/pywikibot/comms/threadedhttp.pyc
-rw-r--r-- 1 framawiki  wikidev           22419 Jul 23 15:44 ./public_html/core/pywikibot/config2.pyc
-rw-r--r-- 1 framawiki  wikidev            1893 Jul 23 15:44 ./public_html/core/pywikibot/daemonize.pyc
-rw-r--r-- 1 framawiki  wikidev          117085 Jul 23 15:44 ./public_html/core/pywikibot/data/api.pyc
-rw-r--r-- 1 framawiki  wikidev             264 Jul 23 15:44 ./public_html/core/pywikibot/data/__init__.pyc
-rw-r--r-- 1 framawiki  wikidev           23094 Jul 23 15:44 ./public_html/core/pywikibot/diff.pyc
-rw-r--r-- 1 framawiki  wikidev            2232 Jul 23 15:44 ./public_html/core/pywikibot/echo.pyc
-rw-r--r-- 1 framawiki  wikidev           24278 Jul 23 15:44 ./public_html/core/pywikibot/exceptions.pyc
-rw-r--r-- 1 framawiki  wikidev           11262 Jul 23 15:44 ./public_html/core/pywikibot/families/wikipedia_family.pyc
-rw-r--r-- 1 framawiki  wikidev           58372 Jul 23 15:44 ./public_html/core/pywikibot/family.pyc
-rw-r--r-- 1 framawiki  wikidev           23415 Jul 23 15:44 ./public_html/core/pywikibot/i18n.pyc
-rw-r--r-- 1 framawiki  wikidev           56419 Jul 23 15:44 ./public_html/core/pywikibot/__init__.pyc
-rw-r--r-- 1 framawiki  wikidev            6899 Jul 23 15:44 ./public_html/core/pywikibot/logging.pyc
-rw-r--r-- 1 framawiki  wikidev           17007 Jul 23 15:44 ./public_html/core/pywikibot/login.pyc
-rw-r--r-- 1 framawiki  wikidev          225321 Jul 23 15:44 ./public_html/core/pywikibot/page.pyc
-rw-r--r-- 1 framawiki  wikidev           12358 Jul 23 15:44 ./public_html/core/pywikibot/plural.pyc
-rw-r--r-- 1 framawiki  wikidev          303384 Jul 23 15:44 ./public_html/core/pywikibot/site.pyc
-rw-r--r-- 1 framawiki  wikidev           69977 Jul 23 15:44 ./public_html/core/pywikibot/textlib.pyc
-rw-r--r-- 1 framawiki  wikidev            9470 Jul 23 15:44 ./public_html/core/pywikibot/throttle.pyc
-rw-r--r-- 1 framawiki  wikidev            3954 Jul 23 15:44 ./public_html/core/pywikibot/tools/chars.pyc
-rw-r--r-- 1 framawiki  wikidev            7917 Jul 23 15:44 ./public_html/core/pywikibot/tools/formatter.pyc
-rw-r--r-- 1 framawiki  wikidev           70060 Jul 23 15:44 ./public_html/core/pywikibot/tools/__init__.pyc
-rw-r--r-- 1 framawiki  wikidev            3081 Jul 23 15:44 ./public_html/core/pywikibot/tools/ip.pyc
-rw-r--r-- 1 framawiki  wikidev            5377 Jul 23 15:44 ./public_html/core/pywikibot/tools/_logging.pyc
-rw-r--r-- 1 framawiki  wikidev             231 Jul 23 15:44 ./public_html/core/pywikibot/userinterfaces/__init__.pyc
-rw-r--r-- 1 framawiki  wikidev           19535 Jul 23 15:44 ./public_html/core/pywikibot/userinterfaces/terminal_interface_base.pyc
-rw-r--r-- 1 framawiki  wikidev             741 Jul 23 15:44 ./public_html/core/pywikibot/userinterfaces/terminal_interface.pyc
-rw-r--r-- 1 framawiki  wikidev            3332 Jul 23 15:44 ./public_html/core/pywikibot/userinterfaces/terminal_interface_unix.pyc
-rw-r--r-- 1 framawiki  wikidev           75834 Jul 23 15:44 ./public_html/core/pywikibot/userinterfaces/transliteration.pyc
-rw-r--r-- 1 framawiki  wikidev           19456 Jul 23 15:44 ./public_html/core/pywikibot/version.pyc
-rw-r--r-- 1 framawiki  wikidev            3788 Jul 23 15:44 ./public_html/core/pywikibot/_wbtypes.pyc
-rw-r--r-- 1 framawiki  wikidev             258 Jul 23 15:45 ./public_html/core/scripts/__init__.pyc
drwxr-xr-x 2 framawiki  wikidev            4096 Jul 13 15:36 ./public_html.old/core/mwparserfromhell/nodes/extras/__pycache__
-rw-r--r-- 1 framawiki  wikidev            4904 Jul 13 15:36 ./public_html.old/core/mwparserfromhell/nodes/extras/__pycache__/attribute.cpython-34.pyc
-rw-r--r-- 1 framawiki  wikidev             458 Jul 13 15:36 ./public_html.old/core/mwparserfromhell/nodes/extras/__pycache__/__init__.cpython-34.pyc
-rw-r--r-- 1 framawiki  wikidev            2767 Jul 13 15:36 ./public_html.old/core/mwparserfromhell/nodes/extras/__pycache__/parameter.cpython-34.pyc
drwxr-xr-x 2 framawiki  wikidev            4096 Jul 13 15:36 ./public_html.old/core/mwparserfromhell/nodes/__pycache__
-rw-r--r-- 1 framawiki  wikidev            2640 Jul 13 15:36 ./public_html.old/core/mwparserfromhell/nodes/__pycache__/argument.cpython-34.pyc
-rw-r--r-- 1 framawiki  wikidev            1263 Jul 13 15:36 ./public_html.old/core/mwparserfromhell/nodes/__pycache__/comment.cpython-34.pyc
-rw-r--r-- 1 framawiki  wikidev            2875 Jul 13 15:36 ./public_html.old/core/mwparserfromhell/nodes/__pycache__/external_link.cpython-34.pyc
-rw-r--r-- 1 framawiki  wikidev            2221 Jul 13 15:36 ./public_html.old/core/mwparserfromhell/nodes/__pycache__/heading.cpython-34.pyc
-rw-r--r-- 1 framawiki  wikidev            5172 Jul 13 15:36 ./public_html.old/core/mwparserfromhell/nodes/__pycache__/html_entity.cpython-34.pyc
-rw-r--r-- 1 framawiki  wikidev            2770 Jul 13 15:36 ./public_html.old/core/mwparserfromhell/nodes/__pycache__/__init__.cpython-34.pyc
-rw-r--r-- 1 framawiki  wikidev           10527 Jul 13 15:36 ./public_html.old/core/mwparserfromhell/nodes/__pycache__/tag.cpython-34.pyc
-rw-r--r-- 1 framawiki  wikidev           12874 Jul 13 15:36 ./public_html.old/core/mwparserfromhell/nodes/__pycache__/template.cpython-34.pyc
-rw-r--r-- 1 framawiki  wikidev            1524 Jul 13 15:36 ./public_html.old/core/mwparserfromhell/nodes/__pycache__/text.cpython-34.pyc
-rw-r--r-- 1 framawiki  wikidev            2363 Jul 13 15:36 ./public_html.old/core/mwparserfromhell/nodes/__pycache__/wikilink.cpython-34.pyc
drwxr-xr-x 2 framawiki  wikidev            4096 Jul 13 15:36 ./public_html.old/core/mwparserfromhell/parser/__pycache__
-rw-r--r-- 1 framawiki  wikidev            9789 Jul 13 15:36 ./public_html.old/core/mwparserfromhell/parser/__pycache__/builder.cpython-34.pyc
-rw-r--r-- 1 framawiki  wikidev            4228 Jul 13 15:36 ./public_html.old/core/mwparserfromhell/parser/__pycache__/contexts.cpython-34.pyc
-rw-r--r-- 1 framawiki  wikidev            3479 Jul 13 15:36 ./public_html.old/core/mwparserfromhell/parser/__pycache__/__init__.cpython-34.pyc
-rw-r--r-- 1 framawiki  wikidev           44997 Jul 13 15:36 ./public_html.old/core/mwparserfromhell/parser/__pycache__/tokenizer.cpython-34.pyc
-rw-r--r-- 1 framawiki  wikidev            3369 Jul 13 15:36 ./public_html.old/core/mwparserfromhell/parser/__pycache__/tokens.cpython-34.pyc
drwxr-xr-x 2 framawiki  wikidev            4096 Jul 13 15:36 ./public_html.old/core/mwparserfromhell/__pycache__
-rw-r--r-- 1 framawiki  wikidev             906 Jul 13 15:36 ./public_html.old/core/mwparserfromhell/__pycache__/compat.cpython-34.pyc
-rw-r--r-- 1 framawiki  wikidev            2563 Jul 13 15:36 ./public_html.old/core/mwparserfromhell/__pycache__/definitions.cpython-34.pyc
-rw-r--r-- 1 framawiki  wikidev             850 Jul 13 15:36 ./public_html.old/core/mwparserfromhell/__pycache__/__init__.cpython-34.pyc
-rw-r--r-- 1 framawiki  wikidev           16643 Jul 13 15:36 ./public_html.old/core/mwparserfromhell/__pycache__/smart_list.cpython-34.pyc
-rw-r--r-- 1 framawiki  wikidev            4694 Jul 13 15:36 ./public_html.old/core/mwparserfromhell/__pycache__/string_mixin.cpython-34.pyc
-rw-r--r-- 1 framawiki  wikidev            2142 Jul 13 15:36 ./public_html.old/core/mwparserfromhell/__pycache__/utils.cpython-34.pyc
-rw-r--r-- 1 framawiki  wikidev           29035 Jul 13 15:36 ./public_html.old/core/mwparserfromhell/__pycache__/wikicode.cpython-34.pyc
drwxr-xr-x 2 framawiki  wikidev            4096 Jul 13 15:36 ./public_html.old/core/pywikibot/comms/__pycache__
-rw-r--r-- 1 framawiki  wikidev           14513 Jul 13 15:36 ./public_html.old/core/pywikibot/comms/__pycache__/http.cpython-34.pyc
-rw-r--r-- 1 framawiki  wikidev             213 Jul 13 15:36 ./public_html.old/core/pywikibot/comms/__pycache__/__init__.cpython-34.pyc
-rw-r--r-- 1 framawiki  wikidev            5851 Jul 13 15:36 ./public_html.old/core/pywikibot/comms/__pycache__/threadedhttp.cpython-34.pyc
drwxr-xr-x 2 framawiki  wikidev            4096 Jul 13 15:36 ./public_html.old/core/pywikibot/data/__pycache__
-rw-r--r-- 1 framawiki  wikidev          100751 Jul 13 15:36 ./public_html.old/core/pywikibot/data/__pycache__/api.cpython-34.pyc
-rw-r--r-- 1 framawiki  wikidev             251 Jul 13 15:36 ./public_html.old/core/pywikibot/data/__pycache__/__init__.cpython-34.pyc
drwxr-xr-x 2 framawiki  wikidev            4096 Jul 13 15:36 ./public_html.old/core/pywikibot/families/__pycache__
-rw-r--r-- 1 framawiki  wikidev            8005 Jul 13 15:36 ./public_html.old/core/pywikibot/families/__pycache__/wikipedia_family.cpython-34.pyc
drwxr-xr-x 2 framawiki  wikidev            4096 Jul 13 15:36 ./public_html.old/core/pywikibot/__pycache__
-rw-r--r-- 1 framawiki  wikidev           13041 Jul 13 15:36 ./public_html.old/core/pywikibot/__pycache__/bot_choice.cpython-34.pyc
-rw-r--r-- 1 framawiki  wikidev           69705 Jul 13 15:36 ./public_html.old/core/pywikibot/__pycache__/bot.cpython-34.pyc
-rw-r--r-- 1 framawiki  wikidev           19538 Jul 13 15:36 ./public_html.old/core/pywikibot/__pycache__/config2.cpython-34.pyc
-rw-r--r-- 1 framawiki  wikidev            1657 Jul 13 15:36 ./public_html.old/core/pywikibot/__pycache__/daemonize.cpython-34.pyc
-rw-r--r-- 1 framawiki  wikidev           19752 Jul 13 15:36 ./public_html.old/core/pywikibot/__pycache__/diff.cpython-34.pyc
-rw-r--r-- 1 framawiki  wikidev            1784 Jul 13 15:36 ./public_html.old/core/pywikibot/__pycache__/echo.cpython-34.pyc
-rw-r--r-- 1 framawiki  wikidev           19318 Jul 13 15:36 ./public_html.old/core/pywikibot/__pycache__/exceptions.cpython-34.pyc
-rw-r--r-- 1 framawiki  wikidev           48413 Jul 13 15:36 ./public_html.old/core/pywikibot/__pycache__/family.cpython-34.pyc
-rw-r--r-- 1 framawiki  wikidev           20825 Jul 13 15:36 ./public_html.old/core/pywikibot/__pycache__/i18n.cpython-34.pyc
-rw-r--r-- 1 framawiki  wikidev           47547 Jul 13 15:36 ./public_html.old/core/pywikibot/__pycache__/__init__.cpython-34.pyc
-rw-r--r-- 1 framawiki  wikidev            5630 Jul 13 15:36 ./public_html.old/core/pywikibot/__pycache__/logging.cpython-34.pyc
-rw-r--r-- 1 framawiki  wikidev           15062 Jul 13 15:36 ./public_html.old/core/pywikibot/__pycache__/login.cpython-34.pyc
-rw-r--r-- 1 framawiki  wikidev          196511 Jul 13 15:36 ./public_html.old/core/pywikibot/__pycache__/page.cpython-34.pyc
-rw-r--r-- 1 framawiki  wikidev            8435 Jul 13 15:36 ./public_html.old/core/pywikibot/__pycache__/plural.cpython-34.pyc
-rw-r--r-- 1 framawiki  wikidev          268056 Jul 13 15:36 ./public_html.old/core/pywikibot/__pycache__/site.cpython-34.pyc
-rw-r--r-- 1 framawiki  wikidev           62162 Jul 13 15:36 ./public_html.old/core/pywikibot/__pycache__/textlib.cpython-34.pyc
-rw-r--r-- 1 framawiki  wikidev            8123 Jul 13 15:36 ./public_html.old/core/pywikibot/__pycache__/throttle.cpython-34.pyc
-rw-r--r-- 1 framawiki  wikidev           16832 Jul 13 15:36 ./public_html.old/core/pywikibot/__pycache__/version.cpython-34.pyc
-rw-r--r-- 1 framawiki  wikidev            2750 Jul 13 15:36 ./public_html.old/core/pywikibot/__pycache__/_wbtypes.cpython-34.pyc
drwxr-xr-x 2 framawiki  wikidev            4096 Jul 13 15:36 ./public_html.old/core/pywikibot/tools/__pycache__
-rw-r--r-- 1 framawiki  wikidev            3436 Jul 13 15:36 ./public_html.old/core/pywikibot/tools/__pycache__/chars.cpython-34.pyc
-rw-r--r-- 1 framawiki  wikidev            6721 Jul 13 15:36 ./public_html.old/core/pywikibot/tools/__pycache__/formatter.cpython-34.pyc
-rw-r--r-- 1 framawiki  wikidev           58616 Jul 13 15:36 ./public_html.old/core/pywikibot/tools/__pycache__/__init__.cpython-34.pyc
-rw-r--r-- 1 framawiki  wikidev            2641 Jul 13 15:36 ./public_html.old/core/pywikibot/tools/__pycache__/ip.cpython-34.pyc
-rw-r--r-- 1 framawiki  wikidev            4690 Jul 13 15:36 ./public_html.old/core/pywikibot/tools/__pycache__/_logging.cpython-34.pyc
drwxr-xr-x 2 framawiki  wikidev            4096 Jul 13 15:36 ./public_html.old/core/pywikibot/userinterfaces/__pycache__
-rw-r--r-- 1 framawiki  wikidev             218 Jul 13 15:36 ./public_html.old/core/pywikibot/userinterfaces/__pycache__/__init__.cpython-34.pyc
-rw-r--r-- 1 framawiki  wikidev           16294 Jul 13 15:36 ./public_html.old/core/pywikibot/userinterfaces/__pycache__/terminal_interface_base.cpython-34.pyc
-rw-r--r-- 1 framawiki  wikidev             675 Jul 13 15:36 ./public_html.old/core/pywikibot/userinterfaces/__pycache__/terminal_interface.cpython-34.pyc
-rw-r--r-- 1 framawiki  wikidev            2557 Jul 13 15:36 ./public_html.old/core/pywikibot/userinterfaces/__pycache__/terminal_interface_unix.cpython-34.pyc
-rw-r--r-- 1 framawiki  wikidev           73264 Jul 13 15:36 ./public_html.old/core/pywikibot/userinterfaces/__pycache__/transliteration.cpython-34.pyc
-rw-r--r-- 1 valhallasw tools.pywikibot  136315 Aug 20  2015 ./rewrite/pywikibot/site.pyc
-rw-r--r-- 1 valhallasw tools.pywikibot 2955312 Jun 18  2013 ./svn2git/libQtCore.so.4
drwxrwxr-x 2 valhallasw             550    4096 Jun 23  2013 ./svn2git/src2
-rw-rw-r-- 1 valhallasw             550   13027 Jun 23  2013 ./svn2git/src2/CommandLineParser.cpp
-rw-rw-r-- 1 valhallasw             550    3656 Jun 23  2013 ./svn2git/src2/CommandLineParser.h
-rw-rw-r-- 1 valhallasw             550    9444 Jun 23  2013 ./svn2git/src2/main.cpp
-rw-r--r-- 1 valhallasw             550    8603 Jun 23  2013 ./svn2git/src2/Makefile
-rw-r--r-- 1 valhallasw             550   30260 Jun 23  2013 ./svn2git/src2/repository.cpp
-rw-rw-r-- 1 valhallasw             550    5925 Jun 23  2013 ./svn2git/src2/repository.h
-rw-rw-r-- 1 valhallasw             550   13325 Jun 23  2013 ./svn2git/src2/ruleparser.cpp
-rw-rw-r-- 1 valhallasw             550    3650 Jun 23  2013 ./svn2git/src2/ruleparser.h
-rw-rw-r-- 1 valhallasw             550    1135 Jun 23  2013 ./svn2git/src2/src.pro
-rw-rw-r-- 1 valhallasw             550   31262 Jun 23  2013 ./svn2git/src2/svn.cpp
-rw-rw-r-- 1 valhallasw             550    1290 Jun 23  2013 ./svn2git/src2/svn.h
-rwxr-xr-x 1 valhallasw tools.pywikibot     959 Jun 23  2013 ./svn2git/takewrapper.py
-rwxr-xr-x 1 valhallasw tools.pywikibot 2070343 Jun 22  2013 ./svn-all-fast-export
root@tools-bastion-05:~# chown -R tools.pywikibot:tools.pywikibot /data/project/pywikibot/public_html{,.old}/core/
tools.pywikibot@tools-bastion-05:~$ find . \! -user tools.pywikibot | xargs ls -ld
-rw-r--r-- 1 root       tools.pywikibot     988 Mar 18  2014 ./...DATA.crontab
-rw-r--r-- 1 ladsgroup  tools.pywikibot    1008 Mar 28  2014 ./gitlog.py
-rw-r--r-- 1 valhallasw tools.pywikibot  152658 May  1  2014 ./public_html.20160522/125575_new.txt
-rw-r--r-- 1 valhallasw tools.pywikibot  155682 May  1  2014 ./public_html.20160522/125575_old.txt
-rw-r--r-- 1 valhallasw tools.pywikibot      92 Nov 20  2014 ./public_html.20160522/tests/upload.php
-rw-r--r-- 1 valhallasw tools.pywikibot  136315 Aug 20  2015 ./rewrite/pywikibot/site.pyc
-rw-r--r-- 1 valhallasw tools.pywikibot 2955312 Jun 18  2013 ./svn2git/libQtCore.so.4
drwxrwxr-x 2 valhallasw             550    4096 Jun 23  2013 ./svn2git/src2
-rw-rw-r-- 1 valhallasw             550   13027 Jun 23  2013 ./svn2git/src2/CommandLineParser.cpp
-rw-rw-r-- 1 valhallasw             550    3656 Jun 23  2013 ./svn2git/src2/CommandLineParser.h
-rw-rw-r-- 1 valhallasw             550    9444 Jun 23  2013 ./svn2git/src2/main.cpp
-rw-r--r-- 1 valhallasw             550    8603 Jun 23  2013 ./svn2git/src2/Makefile
-rw-r--r-- 1 valhallasw             550   30260 Jun 23  2013 ./svn2git/src2/repository.cpp
-rw-rw-r-- 1 valhallasw             550    5925 Jun 23  2013 ./svn2git/src2/repository.h
-rw-rw-r-- 1 valhallasw             550   13325 Jun 23  2013 ./svn2git/src2/ruleparser.cpp
-rw-rw-r-- 1 valhallasw             550    3650 Jun 23  2013 ./svn2git/src2/ruleparser.h
-rw-rw-r-- 1 valhallasw             550    1135 Jun 23  2013 ./svn2git/src2/src.pro
-rw-rw-r-- 1 valhallasw             550   31262 Jun 23  2013 ./svn2git/src2/svn.cpp
-rw-rw-r-- 1 valhallasw             550    1290 Jun 23  2013 ./svn2git/src2/svn.h
-rwxr-xr-x 1 valhallasw tools.pywikibot     959 Jun 23  2013 ./svn2git/takewrapper.py
-rwxr-xr-x 1 valhallasw tools.pywikibot 2070343 Jun 22  2013 ./svn-all-fast-export

Not sure if the rest should be chowned, but since they don't affect this task, meh.

Stashbot added a comment.Jul 29 2018, 4:20 AM

Mentioned in SAL (#wikimedia-cloud) [2018-07-29T04:20:01Z] <zhuyifei1999_> force a run of nightly script T200600

zhuyifei1999 closed this task as Resolved.Jul 29 2018, 4:38 AM
zhuyifei1999 claimed this task.

P7398 pywikibot nightly script run for T200600
1tools.pywikibot@tools-bastion-05:~$ /data/project/pywikibot/pre_nightly
2Cloning into 'nightly-source'...
3remote: Counting objects: 56, done.
4remote: Total 56 (delta 0), reused 0 (delta 0), pack-reused 55
5Unpacking objects: 100% (56/56), done.
6Checking connectivity... done.
7tools.pywikibot@tools-bastion-05:~$ /data/project/pywikibot/nightly-source/nightly
8Building new public_html in /tmp/tmp.fD9fTFa2mj on tools-bastion-05...
9Step 1: mwparserfromhell
10Step 2: compat
11Submodule 'externals/opencv' (https://gerrit.wikimedia.org/r/pywikibot/opencv.git) registered for path 'externals/opencv'
12Submodule 'externals/pycolorname' (https://gerrit.wikimedia.org/r/pywikibot/pycolorname.git) registered for path 'externals/pycolorname'
13Submodule 'externals/spelling' (https://gerrit.wikimedia.org/r/p/pywikibot/spelling.git) registered for path 'externals/spelling'
14Submodule 'i18n' (https://gerrit.wikimedia.org/r/p/pywikibot/i18n.git) registered for path 'i18n'
15Cloning into 'externals/opencv'...
16remote: Total 35 (delta 0), reused 35 (delta 0)
17Unpacking objects: 100% (35/35), done.
18Checking connectivity... done.
19Submodule path 'externals/opencv': checked out '1d51dbbe74a6453da62c6d93fb3305dfebc863eb'
20Cloning into 'externals/pycolorname'...
21remote: Total 22 (delta 0), reused 22 (delta 0)
22Unpacking objects: 100% (22/22), done.
23Checking connectivity... done.
24Submodule path 'externals/pycolorname': checked out '53a900e86b1d04d865f11a1bc4204659c10d540d'
25Cloning into 'externals/spelling'...
26remote: Total 69 (delta 0), reused 69 (delta 0)
27Unpacking objects: 100% (69/69), done.
28Checking connectivity... done.
29Submodule path 'externals/spelling': checked out '203a3b728446466a44764bde916a3c395f564824'
30Cloning into 'i18n'...
31remote: Total 17686 (delta 0), reused 17686 (delta 0)
32Receiving objects: 100% (17686/17686), 13.48 MiB | 6.13 MiB/s, done.
33Resolving deltas: 100% (13839/13839), done.
34Checking connectivity... done.
35Submodule path 'i18n': checked out '6e7e103953a1e0033a8fc0c25aed8582df506322'
36Step 3: core master
37Step 4: core-7 days old
38Switched to a new branch 'core_old'
39Step 5: zip ALL the things
40Step 6: tar ALL the things
41Step 7: copy static files
42Step 8: do the magic switcharoo
43BASH=/bin/bash
44BASHOPTS=cmdhist:complete_fullquote:extquote:force_fignore:hostcomplete:interactive_comments:progcomp:promptvars:sourcepath
45BASH_ALIASES=()
46BASH_ARGC=()
47BASH_ARGV=()
48BASH_CMDS=()
49BASH_LINENO=([0]="0")
50BASH_SOURCE=([0]="/data/project/pywikibot/nightly-source/nightly")
51BASH_VERSINFO=([0]="4" [1]="3" [2]="11" [3]="1" [4]="release" [5]="x86_64-pc-linux-gnu")
52BASH_VERSION='4.3.11(1)-release'
53DIRSTACK=()
54EUID=51183
55GROUPS=()
56HOME=/data/project/pywikibot
57HOSTNAME=tools-bastion-05
58HOSTTYPE=x86_64
59IFS=$' \t\n'
60INSTANCENAME=tools-bastion-05
61INSTANCEPROJECT=tools
62LANG=en_US.UTF-8
63LOGNAME=tools.pywikibot
64LS_COLORS='rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arj=01;31:*.taz=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lz=01;31:*.xz=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.axv=01;35:*.anx=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.axa=00;36:*.oga=00;36:*.spx=00;36:*.xspf=00;36:'
65MACHTYPE=x86_64-pc-linux-gnu
66MAIL=/var/mail/tools.pywikibot
67OLDPWD=/data/project/pywikibot
68OPTERR=1
69OPTIND=1
70OSTYPE=linux-gnu
71PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
72PIPESTATUS=([0]="0")
73PPID=23774
74PS4='+ '
75PWD=/tmp/tmp.fD9fTFa2mj
76SHELL=/bin/bash
77SHELLOPTS=braceexpand:errexit:hashall:interactive-comments:nounset
78SHLVL=2
79SUDO_COMMAND=/bin/bash
80SUDO_GID=500
81SUDO_UID=3306
82SUDO_USER=zhuyifei1999
83TERM=xterm-256color
84TMPDIR=/tmp/tmp.fD9fTFa2mj
85UID=51183
86USER=tools.pywikibot
87USERNAME=tools.pywikibot
88VIRTUALENVWRAPPER_SCRIPT=/usr/share/virtualenvwrapper/virtualenvwrapper.sh
89_=switcharoo
90_VIRTUALENVWRAPPER_API=' mkvirtualenv rmvirtualenv lsvirtualenv showvirtualenv workon add2virtualenv cdsitepackages cdvirtualenv lssitepackages toggleglobalsitepackages cpvirtualenv setvirtualenvproject mkproject cdproject mktmpenv'
91create_version ()
92{
93 echo "$1" > $2;
94 git rev-list HEAD | wc -l >> $2;
95 date +%Y-%m-%dT%H:%M:%S >> $2;
96 git rev-parse --short HEAD >> $2
97}
98finish ()
99{
100 rm -rf "$TMPDIR";
101 echo Cleaned up $TMPDIR
102}
103Cleaned up /tmp/tmp.fD9fTFa2mj

tools.pywikibot@tools-bastion-05:~$ ls public_html -l
total 74180
drwxr-xr-x 24 tools.pywikibot tools.pywikibot     4096 Jul 29 04:19 compat
-rw-r--r--  1 tools.pywikibot tools.pywikibot 18649370 Jul 29 04:23 compat.tar.gz
-rw-r--r--  1 tools.pywikibot tools.pywikibot 20043657 Jul 29 04:22 compat.zip
drwxr-xr-x  8 tools.pywikibot tools.pywikibot     4096 Jul 29 04:20 core
drwxr-xr-x  8 tools.pywikibot tools.pywikibot     4096 Jul 29 04:22 core_old
-rw-r--r--  1 tools.pywikibot tools.pywikibot 12236253 Jul 29 04:22 core_old.tar.gz
-rw-r--r--  1 tools.pywikibot tools.pywikibot 14048521 Jul 29 04:22 core_old.zip
-rw-r--r--  1 tools.pywikibot tools.pywikibot  4590888 Jul 29 04:22 core.tar.gz
-rw-r--r--  1 tools.pywikibot tools.pywikibot  6359582 Jul 29 04:22 core.zip
-rw-r--r--  1 tools.pywikibot tools.pywikibot     3620 Jul 29 04:23 index.html
drwxr-xr-x  7 tools.pywikibot tools.pywikibot     4096 Jul 29 04:19 mwparserfromhell
tools.pywikibot@tools-bastion-05:~$ ls public_html/core -l
total 188
-rw-r--r-- 1 tools.pywikibot tools.pywikibot 21140 Jul 29 04:20 ChangeLog
-rw-r--r-- 1 tools.pywikibot tools.pywikibot   152 Jul 29 04:20 CODE_OF_CONDUCT.rst
-rw-r--r-- 1 tools.pywikibot tools.pywikibot  4686 Jul 29 04:20 CONTENT.rst
-rw-r--r-- 1 tools.pywikibot tools.pywikibot  2133 Jul 29 04:20 CREDITS
-rw-r--r-- 1 tools.pywikibot tools.pywikibot  1013 Jul 29 04:20 dev-requirements.txt
-rw-r--r-- 1 tools.pywikibot tools.pywikibot   598 Jul 29 04:20 Dockerfile
drwxr-xr-x 4 tools.pywikibot tools.pywikibot  4096 Jul 29 04:20 docs
-rw-r--r-- 1 tools.pywikibot tools.pywikibot 12571 Jul 29 04:20 ez_setup.py
-rwxr-xr-x 1 tools.pywikibot tools.pywikibot  7063 Jul 29 04:20 generate_family_file.py
-rwxr-xr-x 1 tools.pywikibot tools.pywikibot 17586 Jul 29 04:20 generate_user_files.py
-rw-r--r-- 1 tools.pywikibot tools.pywikibot  5584 Jul 29 04:20 HISTORY.rst
-rw-r--r-- 1 tools.pywikibot tools.pywikibot  1063 Jul 29 04:20 LICENSE
drwxr-xr-x 4 tools.pywikibot tools.pywikibot  4096 Jul 29 04:20 mwparserfromhell
-rwxr-xr-x 1 tools.pywikibot tools.pywikibot  8800 Jul 29 04:20 pwb.py
drwxr-xr-x 8 tools.pywikibot tools.pywikibot  4096 Jul 29 04:20 pywikibot
-rw-r--r-- 1 tools.pywikibot tools.pywikibot  6931 Jul 29 04:20 README-conversion.txt
-rw-r--r-- 1 tools.pywikibot tools.pywikibot  2665 Jul 29 04:20 README.rst
-rw-r--r-- 1 tools.pywikibot tools.pywikibot   478 Jul 29 04:20 requests-requirements.txt
-rw-r--r-- 1 tools.pywikibot tools.pywikibot  2519 Jul 29 04:20 requirements.txt
drwxr-xr-x 6 tools.pywikibot tools.pywikibot  4096 Jul 29 04:20 scripts
-rw-r--r-- 1 tools.pywikibot tools.pywikibot  9268 Jul 29 04:20 setup.py
drwxr-xr-x 7 tools.pywikibot tools.pywikibot  4096 Jul 29 04:20 tests
-rw-r--r-- 1 tools.pywikibot tools.pywikibot 10415 Jul 29 04:20 tox.ini
-rw-r--r-- 1 tools.pywikibot tools.pywikibot   200 Jul 29 04:20 user-config.py.sample
-rw-r--r-- 1 tools.pywikibot tools.pywikibot   282 Jul 29 04:20 user-fixes.py.sample
tools.pywikibot@tools-bastion-05:~/public_html/core$ git log
commit d38174766d3bc0a8ea990091924dd58d91b73b37
Author: Framawiki <framawiki@tools.wmflabs.org>
Date:   Sat Jul 28 12:39:48 2018 +0200

    [L10N] Update families files
    
    Change-Id: Ie13ef915bafb0b71b0770fe57297c252db17b202

I also noticed that nightly-source is cloned, then chmod-ed to 777 (chmod -R 777 nightly-source in pre_nightly). I can't think of a reason recursive 777 would ever make sense, so changed that to 755.

Legoktm added a comment.Jul 29 2018, 5:29 AM

Awesome, thanks :)

Framawiki added a comment.Aug 3 2018, 8:44 PM
In T200600#4458885, @Legoktm wrote:

Bunch of stuff like:

rm: cannot remove ‘public_html.old/core/pywikibot/userinterfaces/__pycache__/transliteration.cpython-34.pyc’: Permission denied

OK so:

-rw-r--r-- 1 framawiki wikidev 73264 Jul 13 15:36 transliteration.cpython-34.pyc

???

I'm unable to remove it...did you maybe run the script under your own account instead of using the pwb tool?

Yeah, I tried a few times to run the scripts manually without success. But I just checked, I did launch it as tools.pywikibot user, after entering become pywikibot ?!

Thanks everybody for fixing this.

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL