zhuyifei1999
*Not* Serious business title.

Today

  • Clear sailing ahead.

Tomorrow

  • Clear sailing ahead.

Saturday

  • Clear sailing ahead.

User Details

User Since
Oct 13 2014, 10:19 AM (162 w, 3 d)
Availability
Available
IRC Nick
zhuyifei1999
LDAP User
Zhuyifei1999
MediaWiki User
Zhuyifei1999

There is currently no text in this page. You can search for this page title in other pages, or search the related logs.

Recent Activity

Tue, Nov 21

zhuyifei1999 added a comment to T113193: Detect Wikimedia Toolforge environment in pywikibot.version.

Alternatively you can check for the existence / contents of /etc/wmcs-* (or deprecated /etc/wmflabs*):

05:45:19 0 ✓ zhuyifei1999@tools-bastion-02: ~$ for file in /etc/wmcs* /etc/wmflabs*; do echo ${file}:; cat $file; echo EOF; done
/etc/wmcs-imageversion:
3EOF
/etc/wmcs-instancename:
tools-bastion-02
EOF
/etc/wmcs-project:
tools
EOF
/etc/wmflabs_imageversion:
3EOF
/etc/wmflabs-instancename:
tools-bastion-02
EOF
/etc/wmflabs-project:
tools
EOF
Tue, Nov 21, 5:48 AM · Easy, Google-Code-in-2017, Pywikibot-core

Mon, Nov 20

zhuyifei1999 merged T73989: Installation of pdf2djvu into T130138: Install pdf2djvu for Wikisource DjVu aid.
Mon, Nov 20, 12:41 AM · Patch-For-Review, Toolforge, Cloud-Services
zhuyifei1999 merged task T73989: Installation of pdf2djvu into T130138: Install pdf2djvu for Wikisource DjVu aid.
Mon, Nov 20, 12:41 AM · Wikisource, Tools

Sun, Nov 19

zhuyifei1999 added a comment to T73989: Installation of pdf2djvu.

On which environment should this be installed? It's already on toolforge bastion:

09:50:33 0 ✓ zhuyifei1999@tools-bastion-02: ~$ apt search pdf2djvu
Sorting... Done
Full Text Search... Done
pdf2djvu/trusty,now 0.7.17-3ubuntu2 amd64 [installed]
  PDF to DjVu converter
Sun, Nov 19, 9:52 PM · Wikisource, Tools
zhuyifei1999 renamed T180887: Phabricator is showing a lot of scrollbars on comments (Google Chrome) from Phabricator is showing a lot of scrollbars on comments (Chrome Windows) to Phabricator is showing a lot of scrollbars on comments (Google Chrome).
Sun, Nov 19, 12:35 AM · Phabricator (Upstream), Browser-Support-Google-Chrome, Upstream

Sat, Nov 18

zhuyifei1999 closed T135261: {{REVISION*}} magic words should not display usernames and timestamps for null edits as Resolved.

https://commons.wikimedia.org/wiki/User:Zhuyifei1999/sandbox/2 {{REVISIONTIMESTAMP}} now stays 20171118042628 despite two null-edit attempts. If my tests are wrong or there is still something else to be fixed please reopen.

Sat, Nov 18, 4:28 AM · MW-1.31-release-notes (WMF-deploy-2017-11-07 (1.31.0-wmf.7)), MW-1.28-release (WMF-deploy-2016-06-14_(1.28.0-wmf.6)), Patch-For-Review, MediaWiki-Parser
zhuyifei1999 merged T180830: `setup-tomcat` failing with "range_list containes no elements" error from SGE into T178803: `qsub sync -y` jobs failing on Grid Engine with "range_list containes no elements" error.
Sat, Nov 18, 1:53 AM · cloud-services-team (Kanban), Toolforge, Wikisource
zhuyifei1999 merged task T180830: `setup-tomcat` failing with "range_list containes no elements" error from SGE into T178803: `qsub sync -y` jobs failing on Grid Engine with "range_list containes no elements" error.
Sat, Nov 18, 1:52 AM · Toolforge

Fri, Nov 17

zhuyifei1999 placed T179599: Adoption of tb-dev Red Link Recovery tools up for grabs.

Uh, given https://wikitech.wikimedia.org/wiki/Special:Diff/1775977, I personally don't know if this should be continued.

Fri, Nov 17, 10:04 PM · Toolforge-standards-committee
zhuyifei1999 added a comment to T179599: Adoption of tb-dev Red Link Recovery tools.

I did a backup of the the home directory except the cronout and non-latest db backups, to /home/zhuyifei1999/tb-dev.tar.gz (around 11MiB):

Fri, Nov 17, 9:52 PM · Toolforge-standards-committee
zhuyifei1999 created P6348 tools.tb-dev file backup.
Fri, Nov 17, 9:48 PM
zhuyifei1999 changed the status of T179599: Adoption of tb-dev Red Link Recovery tools from Stalled to Open.
Fri, Nov 17, 9:10 PM · Toolforge-standards-committee
zhuyifei1999 renamed T180830: `setup-tomcat` failing with "range_list containes no elements" error from SGE from seting up tomcat in toolforge: tomcat7-instance-create is not found to `setup-tomcat` failing with "range_list containes no elements" error from SGE.
Fri, Nov 17, 7:57 PM · Toolforge
zhuyifei1999 merged T180831: `setup-tomcat` failing with "range_list containes no elements" error into T180830: `setup-tomcat` failing with "range_list containes no elements" error from SGE.
Fri, Nov 17, 7:56 PM · Toolforge
zhuyifei1999 merged task T180831: `setup-tomcat` failing with "range_list containes no elements" error into T180830: `setup-tomcat` failing with "range_list containes no elements" error from SGE.
Fri, Nov 17, 7:56 PM · Toolforge
zhuyifei1999 added a comment to T180830: `setup-tomcat` failing with "range_list containes no elements" error from SGE.

qrsh -now n does work, but it requires user to manually verify host key:

tools.zhuyifei1999-test@tools-bastion-02:~$ /usr/bin/qrsh -now n -o /dev/null -e /dev/null -q "webgrid-generic" -l h_vmem=512m -l release=trusty -b y -N "setup-$tool" tomcat7-instance-create public_tomcat
The authenticity of host '[tools-webgrid-generic-1402.eqiad.wmflabs]:57612 ([10.68.18.50]:57612)' can't be established.
ECDSA key fingerprint is SHA256:ww/8lz8tvZLhmIPu6m+k4CnBzW8odUxNdy+UDTTejbg.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '[tools-webgrid-generic-1402.eqiad.wmflabs]:57612,[10.68.18.50]:57612' (ECDSA) to the list of known hosts.
You are about to create a Tomcat instance in directory 'public_tomcat'
* New Tomcat instance created in public_tomcat
* You might want to edit default configuration in public_tomcat/conf
* Run public_tomcat/bin/startup.sh to start your Tomcat instance
Fri, Nov 17, 7:51 PM · Toolforge
zhuyifei1999 added a comment to T180830: `setup-tomcat` failing with "range_list containes no elements" error from SGE.

$ qconf -sconf | grep MAX_DYN_EC gives no results, but MAX_DYN_EC is mentioned in sge_conf(5) man page and should have a default of 99, which we should not have ran out.

Fri, Nov 17, 7:40 PM · Toolforge
zhuyifei1999 added a comment to T180830: `setup-tomcat` failing with "range_list containes no elements" error from SGE.

The error message comes from SGE, according to Stack Overflow

Fri, Nov 17, 7:29 PM · Toolforge
zhuyifei1999 added a comment to T180830: `setup-tomcat` failing with "range_list containes no elements" error from SGE.

setup-tomcat basically calls tomcat7-instance-create public_tomcat in a webgrid-generic instance.

Fri, Nov 17, 7:26 PM · Toolforge
zhuyifei1999 reassigned T180002: Create new "MP3 uploaders" user group on Commons from R3609901 to kaldari.
Fri, Nov 17, 4:29 AM · Patch-For-Review, MW-1.31-release-notes (WMF-deploy-2017-11-14 (1.31.0-wmf.8)), Support-and-Safety, Commons, User-notice, Multimedia
zhuyifei1999 reassigned T180002: Create new "MP3 uploaders" user group on Commons from R3609901 to kaldari.
Fri, Nov 17, 4:25 AM · Patch-For-Review, MW-1.31-release-notes (WMF-deploy-2017-11-14 (1.31.0-wmf.8)), Support-and-Safety, Commons, User-notice, Multimedia

Thu, Nov 16

zhuyifei1999 added a subtask for T131561: Tool Labs users .bashrc file does not exist for tools accounts: T180723: Load .bashrc when using become.
Thu, Nov 16, 7:13 PM · Toolforge, Cloud-Services
zhuyifei1999 added a parent task for T180723: Load .bashrc when using become: T131561: Tool Labs users .bashrc file does not exist for tools accounts.
Thu, Nov 16, 7:13 PM · Toolforge
zhuyifei1999 edited projects for T180723: Load .bashrc when using become, added: Toolforge; removed Cloud-VPS.
Thu, Nov 16, 7:03 PM · Toolforge

Wed, Nov 15

zhuyifei1999 added a subtask for T180636: Make Dispenser's principle_links table accessible in new Wiki replica cluster: T173511: Implement technical details and process for "datasets_p" on wikireplica hosts.
Wed, Nov 15, 9:23 PM · Goal, cloud-services-team (FY2017-18), Data-Services, DBA
zhuyifei1999 added a parent task for T173511: Implement technical details and process for "datasets_p" on wikireplica hosts: T180636: Make Dispenser's principle_links table accessible in new Wiki replica cluster.
Wed, Nov 15, 9:23 PM · Data-Services, DBA, cloud-services-team (Kanban), Analytics, Research
zhuyifei1999 added a comment to T180555: Turn the markAdmins gadget into an extension.

Perhaps the easiest solution is to load the commons one on other wikis?

There isn't anything magical here.

Wed, Nov 15, 4:56 AM · Community-Tech, Gadgets

Tue, Nov 14

zhuyifei1999 added a comment to T149290: Enable Huggle on Commons.

Can someone tell me how vandalism is normally dealt with on commons?

Revert & block. See also https://commons.wikimedia.org/wiki/Commons:Vandalism

How many warning levels?

There are, but not often used AFAIK:

Where are vandals being reported to?

https://commons.wikimedia.org/wiki/Commons:Administrators%27_noticeboard/Vandalism

Tue, Nov 14, 8:40 PM · Commons, Huggle
zhuyifei1999 added a project to T180485: Inputbox does not support variants conversion on Chinese sites: Chinese-Sites.
Tue, Nov 14, 4:51 PM · MW-1.31-release-notes (WMF-deploy-2017-11-14 (1.31.0-wmf.8)), Patch-For-Review, Regression, MediaWiki-Language-converter, Chinese-Sites, MediaWiki-extensions-InputBox
zhuyifei1999 merged T180489: Any Chinese Wiki's projects about button or input bar can not auto change to Simplified Chinese or Traditional Chinese into T180485: Inputbox does not support variants conversion on Chinese sites.
Tue, Nov 14, 4:50 PM · MW-1.31-release-notes (WMF-deploy-2017-11-14 (1.31.0-wmf.8)), Patch-For-Review, Regression, MediaWiki-Language-converter, Chinese-Sites, MediaWiki-extensions-InputBox
zhuyifei1999 merged task T180489: Any Chinese Wiki's projects about button or input bar can not auto change to Simplified Chinese or Traditional Chinese into T180485: Inputbox does not support variants conversion on Chinese sites.
Tue, Nov 14, 4:50 PM · Chinese-Sites

Mon, Nov 13

zhuyifei1999 added a comment to T171881: CL support for Wikipedia Zero piracy problems.

What I really need to dig on this further is an easy way to see a list of recent WP0-abuse-related deletions on various wikis. Am I missing some way to use the deletion log search interfaces?

Mon, Nov 13, 10:56 PM · Patch-For-Review, Community-Liaisons (Oct-Dec 2017), Zero

Sat, Nov 11

zhuyifei1999 added a member for Trusted-Contributors: Steinsplitter.
Sat, Nov 11, 6:03 PM

Fri, Nov 10

zhuyifei1999 added a parent task for T180241: Please add images.collection.cooperhewitt.org to $wgCopyUploadsDomains: T60224: Add domains to $wgCopyUploadsDomains (tracking).
Fri, Nov 10, 7:51 PM · Patch-For-Review, Commons, Wikimedia-Site-requests
zhuyifei1999 added a subtask for T60224: Add domains to $wgCopyUploadsDomains (tracking): T180241: Please add images.collection.cooperhewitt.org to $wgCopyUploadsDomains.
Fri, Nov 10, 7:51 PM · Commons, Tracking, Wikimedia-Site-requests

Thu, Nov 9

zhuyifei1999 added a comment to T180141: Raise concurrent mysql connection limit for Quarry (or throttle application concurrency).

There may be as many as 48 celery workers:

1​zhuyifei1999@quarry-runner-01:~$ ps uf -u quarry | tee /dev/stderr | wc -l
2​USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
3​quarry 4169 9.9 0.4 131208 16724 ? Rs Sep26 6402:01 python2.7 /usr/bin/celery worker --app quarry.web.worker --loglevel ERROR
4​quarry 4047 0.0 0.7 138084 31404 ? S Oct31 4:04 \_ python2.7 /usr/bin/celery worker --app quarry.web.worker --loglevel ERROR
5​quarry 27198 0.0 0.6 135624 26328 ? S Nov03 0:07 \_ python2.7 /usr/bin/celery worker --app quarry.web.worker --loglevel ERROR
6​quarry 19152 0.0 1.7 166732 71636 ? S Nov05 0:41 \_ python2.7 /usr/bin/celery worker --app quarry.web.worker --loglevel ERROR
7​quarry 19216 0.0 0.7 133984 30824 ? S Nov05 0:09 \_ python2.7 /usr/bin/celery worker --app quarry.web.worker --loglevel ERROR
8​quarry 25980 0.0 0.7 133848 29996 ? S Nov05 0:06 \_ python2.7 /usr/bin/celery worker --app quarry.web.worker --loglevel ERROR
9​quarry 26600 0.0 0.6 134168 27816 ? S Nov05 0:04 \_ python2.7 /usr/bin/celery worker --app quarry.web.worker --loglevel ERROR
10​quarry 30125 0.0 0.7 132644 28688 ? S Nov05 0:00 \_ python2.7 /usr/bin/celery worker --app quarry.web.worker --loglevel ERROR
11​quarry 8947 0.0 0.9 139020 36580 ? S Nov05 0:12 \_ python2.7 /usr/bin/celery worker --app quarry.web.worker --loglevel ERROR
12​quarry 14681 0.0 0.3 131208 15240 ? S Nov06 0:00 \_ python2.7 /usr/bin/celery worker --app quarry.web.worker --loglevel ERROR
13​quarry 14863 0.0 0.3 131208 15396 ? S Nov06 0:00 \_ python2.7 /usr/bin/celery worker --app quarry.web.worker --loglevel ERROR
14​quarry 14864 0.0 0.3 131208 15492 ? S Nov06 0:00 \_ python2.7 /usr/bin/celery worker --app quarry.web.worker --loglevel ERROR
15​quarry 27711 0.0 0.7 132752 31712 ? S Nov06 0:02 \_ python2.7 /usr/bin/celery worker --app quarry.web.worker --loglevel ERROR
16​quarry 26547 0.0 0.7 132632 31136 ? S Nov08 0:00 \_ python2.7 /usr/bin/celery worker --app quarry.web.worker --loglevel ERROR
17​quarry 26751 0.0 0.7 133268 31608 ? S Nov08 0:01 \_ python2.7 /usr/bin/celery worker --app quarry.web.worker --loglevel ERROR
18​quarry 27558 0.0 0.7 132620 30936 ? S Nov08 0:00 \_ python2.7 /usr/bin/celery worker --app quarry.web.worker --loglevel ERROR
19​quarry 16936 0.0 0.7 132620 31204 ? S 06:19 0:00 \_ python2.7 /usr/bin/celery worker --app quarry.web.worker --loglevel ERROR
20​quarry 17175 0.0 0.3 131208 15944 ? S 06:31 0:00 \_ python2.7 /usr/bin/celery worker --app quarry.web.worker --loglevel ERROR
21​quarry 28278 0.0 0.7 132488 29788 ? S 13:01 0:00 \_ python2.7 /usr/bin/celery worker --app quarry.web.worker --loglevel ERROR
22​quarry 28288 0.0 0.7 132628 30784 ? S 13:01 0:00 \_ python2.7 /usr/bin/celery worker --app quarry.web.worker --loglevel ERROR
23​quarry 28315 0.0 0.7 132232 29616 ? S 13:03 0:00 \_ python2.7 /usr/bin/celery worker --app quarry.web.worker --loglevel ERROR
24​quarry 28346 0.0 0.7 132232 29628 ? S 13:06 0:00 \_ python2.7 /usr/bin/celery worker --app quarry.web.worker --loglevel ERROR
25​quarry 29129 0.0 0.3 131208 15964 ? S 13:31 0:00 \_ python2.7 /usr/bin/celery worker --app quarry.web.worker --loglevel ERROR
26​quarry 29130 0.0 0.3 131208 15960 ? S 13:31 0:00 \_ python2.7 /usr/bin/celery worker --app quarry.web.worker --loglevel ERROR
27​quarry 30654 0.0 0.3 131208 15960 ? S 14:19 0:00 \_ python2.7 /usr/bin/celery worker --app quarry.web.worker --loglevel ERROR
28​26
29​zhuyifei1999@quarry-runner-02:~$ ps uf -u quarry | tee /dev/stderr | wc -l
30​USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
31​quarry 11362 9.5 0.3 134768 15472 ? Ss Sep26 6091:07 python2.7 /usr/bin/celery worker --app quarry.web.worker --loglevel ERROR
32​quarry 32711 0.0 0.0 135024 3104 ? S Oct11 0:02 \_ python2.7 /usr/bin/celery worker --app quarry.web.worker --loglevel ERROR
33​quarry 26399 0.0 0.0 133232 2536 ? S Oct27 0:00 \_ python2.7 /usr/bin/celery worker --app quarry.web.worker --loglevel ERROR
34​quarry 7578 0.0 0.0 133488 2508 ? S Oct29 0:00 \_ python2.7 /usr/bin/celery worker --app quarry.web.worker --loglevel ERROR
35​quarry 26331 0.0 0.0 133488 2512 ? S Oct31 0:00 \_ python2.7 /usr/bin/celery worker --app quarry.web.worker --loglevel ERROR
36​quarry 12424 0.0 0.2 134768 12144 ? S Nov01 0:00 \_ python2.7 /usr/bin/celery worker --app quarry.web.worker --loglevel ERROR
37​quarry 5429 0.0 0.0 134512 2896 ? S Nov02 0:00 \_ python2.7 /usr/bin/celery worker --app quarry.web.worker --loglevel ERROR
38​quarry 23761 0.0 0.0 133488 2516 ? S Nov03 0:00 \_ python2.7 /usr/bin/celery worker --app quarry.web.worker --loglevel ERROR
39​quarry 2546 0.0 0.0 133488 2516 ? S Nov04 0:00 \_ python2.7 /usr/bin/celery worker --app quarry.web.worker --loglevel ERROR
40​quarry 12180 0.0 0.0 134768 2856 ? S Nov06 0:00 \_ python2.7 /usr/bin/celery worker --app quarry.web.worker --loglevel ERROR
41​quarry 32251 0.0 0.0 134768 2592 ? S Nov08 0:00 \_ python2.7 /usr/bin/celery worker --app quarry.web.worker --loglevel ERROR
42​quarry 719 0.0 0.2 134768 8260 ? S Nov08 0:00 \_ python2.7 /usr/bin/celery worker --app quarry.web.worker --loglevel ERROR
43​quarry 7455 0.0 0.5 135024 21868 ? S Nov08 0:01 \_ python2.7 /usr/bin/celery worker --app quarry.web.worker --loglevel ERROR
44​quarry 11830 0.0 0.4 135792 19820 ? S Nov08 0:01 \_ python2.7 /usr/bin/celery worker --app quarry.web.worker --loglevel ERROR
45​quarry 25723 0.0 0.5 135024 22184 ? S 06:17 0:01 \_ python2.7 /usr/bin/celery worker --app quarry.web.worker --loglevel ERROR
46​quarry 25732 0.0 0.5 135024 23512 ? S 06:18 0:01 \_ python2.7 /usr/bin/celery worker --app quarry.web.worker --loglevel ERROR
47​quarry 25742 0.0 0.5 135024 24084 ? S 06:18 0:00 \_ python2.7 /usr/bin/celery worker --app quarry.web.worker --loglevel ERROR
48​quarry 26510 0.0 0.0 134768 2684 ? S 06:30 0:00 \_ python2.7 /usr/bin/celery worker --app quarry.web.worker --loglevel ERROR
49​quarry 26867 0.0 0.6 135024 27964 ? S 06:39 0:01 \_ python2.7 /usr/bin/celery worker --app quarry.web.worker --loglevel ERROR
50​quarry 30998 0.0 0.7 135024 30316 ? S 08:49 0:00 \_ python2.7 /usr/bin/celery worker --app quarry.web.worker --loglevel ERROR
51​quarry 7073 0.0 0.7 135024 30108 ? S 12:58 0:00 \_ python2.7 /usr/bin/celery worker --app quarry.web.worker --loglevel ERROR
52​quarry 7170 0.0 0.7 135024 30208 ? S 13:03 0:00 \_ python2.7 /usr/bin/celery worker --app quarry.web.worker --loglevel ERROR
53​quarry 8173 0.0 0.7 135024 30052 ? S 13:30 0:00 \_ python2.7 /usr/bin/celery worker --app quarry.web.worker --loglevel ERROR
54​quarry 8191 0.0 0.7 134768 29692 ? S 13:31 0:00 \_ python2.7 /usr/bin/celery worker --app quarry.web.worker --loglevel ERROR
55​quarry 8192 0.0 0.3 134768 14672 ? S 13:31 0:00 \_ python2.7 /usr/bin/celery worker --app quarry.web.worker --loglevel ERROR
56​26

Thu, Nov 9, 3:29 PM · Patch-For-Review, DBA, Quarry, Data-Services
zhuyifei1999 created P6298 Quarry celery workers process list.
Thu, Nov 9, 3:28 PM

Tue, Nov 7

zhuyifei1999 added a project to T179985: Cron does not know about kubernetes: Kubernetes.
Tue, Nov 7, 10:43 PM · Kubernetes, Toolforge
zhuyifei1999 added a project to T179967: UploadWizard can overwrite existing files uploaded in the same session before clicking "Upload more files": Commons.
Tue, Nov 7, 7:38 PM · MW-1.31-release-notes (WMF-deploy-2017-11-28 (1.31.0-wmf.10)), Patch-For-Review, Multimedia-Team-Working-Board, Commons, Multimedia, UploadWizard

Sun, Nov 5

zhuyifei1999 added a comment to T179787: upload.wikimedia.org reports wrong mimetype for svg.

See also T150929

Sun, Nov 5, 8:50 PM · Operations, media-storage

Fri, Nov 3

zhuyifei1999 renamed T179737: PAWS HTTP error 500 on https://paws.wmflabs.org/paws/user/pokestarfan/tree from PAWS cannot access to PAWS HTTP error 500 on https://paws.wmflabs.org/paws/user/pokestarfan/tree.
Fri, Nov 3, 11:48 PM · PAWS
zhuyifei1999 added a comment to T179737: PAWS HTTP error 500 on https://paws.wmflabs.org/paws/user/pokestarfan/tree.

What error message(s) do you get when you login? Exact steps to reproduce are welcome.

Fri, Nov 3, 10:04 PM · PAWS
zhuyifei1999 added a comment to T156174: Rewrite /usr/local/bin/crontab in python; fix bugs.
tools.yifeibot@tools-bastion-02:~$ oge-crontab -e
Traceback (most recent call last):
  File "/usr/bin/oge-crontab", line 337, in <module>
    main()
  File "/usr/bin/oge-crontab", line 283, in main
    new_crontab_text = editor(crontab_text)
  File "/usr/bin/oge-crontab", line 209, in editor
    f.write(text.encode('latin1'))
UnicodeEncodeError: 'latin-1' codec can't encode characters in position 1522-1534: ordinal not in range(256)
Fri, Nov 3, 9:30 PM · Patch-For-Review, cloud-services-team (Kanban), Toolforge
zhuyifei1999 changed the status of T179599: Adoption of tb-dev Red Link Recovery tools from Open to Stalled.

Waiting 14 days, in case #tb object to this adoption. In the meantime, Toolforge-standards-committee, anyone object?

Fri, Nov 3, 8:21 PM · Toolforge-standards-committee
zhuyifei1999 added a comment to T179599: Adoption of tb-dev Red Link Recovery tools.

.bash_history copied to P6252

Fri, Nov 3, 2:05 AM · Toolforge-standards-committee

Thu, Nov 2

zhuyifei1999 added a comment to T179599: Adoption of tb-dev Red Link Recovery tools.

List of non-world-readable files/directories:

tools.tb-dev@tools-bastion-05:~$ find ! -perm -g=r | xargs ls -ld --color=auto
-rw------- 1 tools.tb-dev tools.tb-dev  34137 Nov  2 22:23 ./.bash_history
drwx--S--- 2 tools.tb-dev tools.tb-dev   4096 May 14  2014 ./.cache
drwx--S--- 2 tools.tb-dev tools.tb-dev   4096 Jan 21  2017 ./credentials
-r-------- 1 tools.tb-dev tools.tb-dev     52 Jan 21  2017 ./credentials/s51111
drwx--S--- 3 tools.tb-dev tools.tb-dev   4096 May 22  2013 ./.emacs.d
drwx--S--- 2 tools.tb-dev tools.tb-dev   4096 May 22  2013 ./.emacs.d/auto-save-list
-r-------- 1 tools.tb-dev tools.tb-dev    692 Jul  6  2016 ./.kube/config
-rw------- 1 tools.tb-dev tools.tb-dev     41 Jun 24  2013 ./.lesshst
-r-------- 1 tools.tb-dev tools.tb-dev     52 Jan 21  2017 ./.my.cnf
-rw------- 1 tools.tb-dev tools.tb-dev 912398 Nov  2 21:41 ./.mysql_history
-r-------- 1 tools.tb-dev tools.tb-dev     52 Dec 21  2016 ./replica.my.cnf
drwx--S--- 2 tools.tb-dev tools.tb-dev   4096 May  3  2015 ./.ssh
-rw------- 1 tools.tb-dev tools.tb-dev    884 May  3  2015 ./.ssh/known_hosts
drwx--s--x 2 tools.tb-dev tools.tb-dev   4096 Jul  5  2009 ./ts/public_html/cgi-bin
-rw------- 1 tools.tb-dev tools.tb-dev  26716 Jun 24 16:33 ./.viminfo
Thu, Nov 2, 10:59 PM · Toolforge-standards-committee
zhuyifei1999 added a comment to T179599: Adoption of tb-dev Red Link Recovery tools.

I'm unable to find any oauth-related code via tools.tb-dev@tools-bastion-05:~$ find -name \*.txt | grep -i 'oauth'. I don't find any svn or git repos either for this tool, so checking each file manually can be a lot of work.

Thu, Nov 2, 9:53 PM · Toolforge-standards-committee
zhuyifei1999 added a comment to T179599: Adoption of tb-dev Red Link Recovery tools.

The directory /data/project/tb-dev/credentials contains one regular file, s51111, which is exactly the same (binary-wise) as /data/project/tb-dev/replica.my.cnf. Since the latter is immutable, I don't find it necessary to delete that file.

Thu, Nov 2, 9:42 PM · Toolforge-standards-committee
zhuyifei1999 claimed T179599: Adoption of tb-dev Red Link Recovery tools.
Thu, Nov 2, 9:09 PM · Toolforge-standards-committee
zhuyifei1999 added a comment to T172972: Tool "wikidata-timeline" loads Google Analytics.
Has there been any consideration for providing an in-house alternative?

I don't think there is a toolforge in-house alternative, but you may want to implement your own.

Thu, Nov 2, 9:09 PM · Tools
zhuyifei1999 added a comment to T106516: Greyscale pngs without gAMA chunk rendered with incorrect contrast [or setting the gamma in GIMP exports to PNG].

https://commons.wikimedia.org/wiki/File:Esperanza_Barrios.png is another case reported to IRC a few minutes ago.

Thu, Nov 2, 7:12 PM · Multimedia, MediaWiki-File-management, Patch-For-Review, Easy, Upstream, Commons

Wed, Nov 1

zhuyifei1999 edited projects for T179510: Create a new basic ContentAdmin role in Striker, added: Striker; removed Toolforge.
Wed, Nov 1, 10:29 PM · Striker

Sat, Oct 28

zhuyifei1999 awarded T179219: Generic hostnames for wiki database replicas? a Love token.
Sat, Oct 28, 2:49 AM · Data-Services

Fri, Oct 27

zhuyifei1999 added a comment to T178258: https://tools.wmflabs.org/admin/tools layout slightly broken except the first tools in Firefox 56.

For me, it randomly fixes itself and randomly break again at a random position. This random happens whenever a re-render is triggered, such as when the console opens/closes, AFAICT.

Fri, Oct 27, 7:01 PM · Tools, Toolforge
zhuyifei1999 closed T179122: Commons Good pictures "Connecting..." Indefinitely as Invalid.

The functionality is provided by https://commons.wikimedia.org/wiki/Help:FastCCI . It is a gadget.
According to the last item in the dropdown, https://github.com/Commonists/fastcci/issues is the place to report issues with this gadget.
Hence I am closing this task as invalid as the maintainers of FastCCI do not use Wikimedia Phabricator.

Fri, Oct 27, 1:06 AM · Commons

Thu, Oct 26

zhuyifei1999 merged T179103: Formatting of hosted tools page on Toolforge is screwed up into T178258: https://tools.wmflabs.org/admin/tools layout slightly broken except the first tools in Firefox 56.
Thu, Oct 26, 6:35 PM · Tools, Toolforge
zhuyifei1999 merged task T179103: Formatting of hosted tools page on Toolforge is screwed up into T178258: https://tools.wmflabs.org/admin/tools layout slightly broken except the first tools in Firefox 56.
Thu, Oct 26, 6:35 PM · Toolforge
zhuyifei1999 added a watcher for Technical-Tool-Request: zhuyifei1999.
Thu, Oct 26, 3:58 PM

Wed, Oct 25

zhuyifei1999 added a project to T179041: Toolforge oursql connecting to enwiki.analytics.db.svc.eqiad.wmflabs raises error 1615 'Prepared statement needs to be re-prepared' but works fine on enwiki.labsdb: DBA.
Wed, Oct 25, 9:30 PM · Patch-For-Review, DBA, Data-Services
zhuyifei1999 created T179041: Toolforge oursql connecting to enwiki.analytics.db.svc.eqiad.wmflabs raises error 1615 'Prepared statement needs to be re-prepared' but works fine on enwiki.labsdb.
Wed, Oct 25, 9:13 PM · Patch-For-Review, DBA, Data-Services
zhuyifei1999 added a subtask for T60224: Add domains to $wgCopyUploadsDomains (tracking): T178753: Please add <aleph500.biblacad.ro:8991/> to $wgCopyUploadsDomains.
Wed, Oct 25, 6:33 PM · Commons, Tracking, Wikimedia-Site-requests
zhuyifei1999 added a parent task for T178753: Please add <aleph500.biblacad.ro:8991/> to $wgCopyUploadsDomains: T60224: Add domains to $wgCopyUploadsDomains (tracking).
Wed, Oct 25, 6:33 PM · Patch-For-Review, Commons, Wikimedia-Site-requests

Tue, Oct 24

zhuyifei1999 added a parent task for T178959: Retire outdated tools-static resources in favour of CDNJS: T103934: Convert all Labs tools to use cdnjs for static libraries and fonts.
Tue, Oct 24, 11:44 PM · Tools
zhuyifei1999 added a subtask for T103934: Convert all Labs tools to use cdnjs for static libraries and fonts: T178959: Retire outdated tools-static resources in favour of CDNJS.
Tue, Oct 24, 11:44 PM · Community-Tech-Tool-Labs, Tools, Epic
zhuyifei1999 added a subtask for T60224: Add domains to $wgCopyUploadsDomains (tracking): T178961: Please add *.500px.com to the wgCopyUploadsDomains whitelist of Wikimedia Commons.
Tue, Oct 24, 11:03 PM · Commons, Tracking, Wikimedia-Site-requests
zhuyifei1999 added a parent task for T178961: Please add *.500px.com to the wgCopyUploadsDomains whitelist of Wikimedia Commons: T60224: Add domains to $wgCopyUploadsDomains (tracking).
Tue, Oct 24, 11:03 PM · Commons, Wikimedia-Site-requests

Oct 24 2017

zhuyifei1999 added a comment to T178848: Error: BIGINT UNSIGNED value is out of range.

That should correspond to r1.rev_len - r2.rev_len. On a stack overflow post I found two solutions:

select r1.*, cast(r1.rev_len as signed) - cast(r2.rev_len as signed) added_bytes
from revision r1
join revision r2
  on r1.rev_parent_id = r2.rev_id
where
  r1.rev_user_text = 'Tgeik'
  and cast(r1.rev_len as signed) - cast(r2.rev_len as signed) > 1000
limit 100;

or:

SET SESSION sql_mode='NO_UNSIGNED_SUBTRACTION';
select r1.*, r1.rev_len - r2.rev_len added_bytes
from revision r1
join revision r2
  on r1.rev_parent_id = r2.rev_id
where
  r1.rev_user_text = 'Tgeik'
  and r1.rev_len - r2.rev_len > 1000
limit 100

Both works for me.

Oct 24 2017, 5:02 PM · User-bd808, Data-Services

Oct 23 2017

zhuyifei1999 edited projects for T178848: Error: BIGINT UNSIGNED value is out of range, added: Data-Services; removed Quarry, Cloud-Services.
Oct 23 2017, 10:14 PM · User-bd808, Data-Services
zhuyifei1999 added a comment to T135261: {{REVISION*}} magic words should not display usernames and timestamps for null edits.

@zhuyifei1999 This is very desired change, is there anything that can be done?

Oct 23 2017, 10:13 PM · MW-1.31-release-notes (WMF-deploy-2017-11-07 (1.31.0-wmf.7)), MW-1.28-release (WMF-deploy-2016-06-14_(1.28.0-wmf.6)), Patch-For-Review, MediaWiki-Parser
zhuyifei1999 added a project to T178837: Strange Mysql server gone away error - possible outage during 2017-10-22 night: Data-Services.
Oct 23 2017, 8:10 PM · Data-Services, User-Urbanecm, Toolforge, User-bd808

Oct 22 2017

zhuyifei1999 moved T178751: Provide an API so external tools can run SQL queries via Quarry from Backlog to Feature request on the Quarry board.
Oct 22 2017, 8:55 PM · Quarry
zhuyifei1999 added a comment to T108237: Rate-limit media transcode resets in TMH.

Related: T154737

Oct 22 2017, 7:18 PM · TimedMediaHandler-Transcode, Security-Extensions, TimedMediaHandler
zhuyifei1999 edited projects for T178724: Full hd video on edge with black thumbnail, added: TimedMediaHandler; removed TimedMediaHandler-Transcode.
Oct 22 2017, 7:14 PM · TimedMediaHandler, Wikimedia-Video, Commons
zhuyifei1999 added a comment to T178724: Full hd video on edge with black thumbnail.

Might be related to undefined duration?

$ ffprobe Uhr-bahnhof-stadelhofen-zuerich-2016.webm 2>&1 | grep Duration
  Duration: N/A, start: 0.000000, bitrate: N/A
Oct 22 2017, 7:12 PM · TimedMediaHandler, Wikimedia-Video, Commons
zhuyifei1999 added a comment to T178751: Provide an API so external tools can run SQL queries via Quarry.

I'd like to be able to offer a new switch so users can send SQL queries from the command-line with Wikiget via Quarry API

At the current state, I don't think this is within the current scope of Quarry, which provides a means for researchers to share and review each other's queries, rather than command-line automation.

Oct 22 2017, 5:11 AM · Quarry
zhuyifei1999 added a comment to T178751: Provide an API so external tools can run SQL queries via Quarry.

If you are developing a tool or bot, you can run it on Wikimedia Cloud VPS or Toolforge and connect to the wiki replicas directly. Please see https://wikitech.wikimedia.org/wiki/Portal:Data_Services#Wiki_Replicas. I would not recommend using quarry as an machine interface (rather than human usage) to wiki replicas, as the result of every single query run has to be stored somewhere indefinitely.

Oct 22 2017, 3:04 AM · Quarry
zhuyifei1999 renamed T178751: Provide an API so external tools can run SQL queries via Quarry from Quary API to Quarry API.
Oct 22 2017, 2:51 AM · Quarry

Oct 21 2017

zhuyifei1999 added a comment to T165625: Evaluate future of wmf puppet module "mysql".

quarry (it's on trusty and has the precise repos, how does that even work right now?

Yuvi's magic?

Oct 21 2017, 12:29 AM · Icinga, Quarry, Community-Wikimetrics, DBA, Cloud-Services, Operations

Oct 20 2017

zhuyifei1999 added a comment to T178567: Server error (500) while trying to download files from Commons from PAWS.

I don't think thumbor should be involved here. The script fetch original versions of the files, not the thumbnails.

Oct 20 2017, 5:51 PM · Patch-For-Review, media-storage, Operations, Traffic, Pywikibot-Commons, PAWS
zhuyifei1999 added a comment to T178567: Server error (500) while trying to download files from Commons from PAWS.

Uh, now all of the files on that list can be downloaded with pywikibot anyhow

Oct 20 2017, 4:54 PM · Patch-For-Review, media-storage, Operations, Traffic, Pywikibot-Commons, PAWS
zhuyifei1999 added a comment to T178567: Server error (500) while trying to download files from Commons from PAWS.

Changing the script from page.download() to print(page.latest_file_info.url), then in terminal run:

zhuyifei1999@PAWS:~$ (for URL in `python test.py`; do curl -sv $URL 2>&1 > /dev/null; done) | grep 500 -B 10 -A 10
< HTTP/1.1 200 OK
< Date: Fri, 20 Oct 2017 16:51:19 GMT
< Content-Type: image/jpeg
< Content-Length: 760174
< Connection: keep-alive
< X-Object-Meta-Sha1Base36: cl2v7iv5wscse0fuhvmtn84j80qyteh
< Last-Modified: Wed, 18 Oct 2017 18:54:03 GMT
< Etag: 70a159293f426f7549a565a0d6b51c63
< X-Timestamp: 1508352842.49312
< X-Trans-Id: tx985b1935e62c422d833bb-0059ea1796
< X-Varnish: 563787331 554400785, 335008490
< Via: 1.1 varnish-v4, 1.1 varnish-v4
< Accept-Ranges: bytes
< Age: 4593
< X-Cache: cp1074 hit/13, cp1074 pass
< X-Cache-Status: hit
< Strict-Transport-Security: max-age=106384710; includeSubDomains; preload
< X-Analytics: https=1;nocookies=1
< X-Client-IP: 10.68.22.24
< Access-Control-Allow-Origin: *
< Access-Control-Expose-Headers: Age, Date, Content-Length, Content-Range, X-Content-Duration, X-Cache, X-Varnish
--
< HTTP/1.1 200 OK
< Date: Fri, 20 Oct 2017 16:51:20 GMT
< Content-Type: image/jpeg
< Content-Length: 799294
< Connection: keep-alive
< X-Object-Meta-Sha1Base36: 08ixghw48c5zr1kvkvrf89ziuvvfrz4
< Last-Modified: Wed, 18 Oct 2017 18:54:13 GMT
< Etag: 8a5109ee8547ab158f6164eb7cc98eac
< X-Timestamp: 1508352852.76105
< X-Trans-Id: txba3170a03d8740f0a989a-0059ea1799
< X-Varnish: 353193777 350054107, 336128072
< Via: 1.1 varnish-v4, 1.1 varnish-v4
< Accept-Ranges: bytes
< Age: 4590
< X-Cache: cp1071 hit/4, cp1074 pass
< X-Cache-Status: hit
< Strict-Transport-Security: max-age=106384710; includeSubDomains; preload
< X-Analytics: https=1;nocookies=1
< X-Client-IP: 10.68.22.24
< Access-Control-Allow-Origin: *
< Access-Control-Expose-Headers: Age, Date, Content-Length, Content-Range, X-Content-Duration, X-Cache, X-Varnish
--
< HTTP/1.1 200 OK
< Date: Fri, 20 Oct 2017 16:51:20 GMT
< Content-Type: image/jpeg
< Content-Length: 800212
< Connection: keep-alive
< X-Object-Meta-Sha1Base36: 1vd82971shruich1pbbd0eqrnbl2oe2
< Last-Modified: Wed, 18 Oct 2017 18:54:44 GMT
< Etag: 167243d3a1afb0a5b471e54cba18300e
< X-Timestamp: 1508352883.57461
< X-Trans-Id: txe5919f4c19e14095a4736-0059ea179a
< X-Varnish: 563005766 560935005, 341150921
< Via: 1.1 varnish-v4, 1.1 varnish-v4
< Accept-Ranges: bytes
< Age: 4589
< X-Cache: cp1074 hit/12, cp1074 pass
< X-Cache-Status: hit
< Strict-Transport-Security: max-age=106384710; includeSubDomains; preload
< X-Analytics: https=1;nocookies=1
< X-Client-IP: 10.68.22.24
< Access-Control-Allow-Origin: *
< Access-Control-Expose-Headers: Age, Date, Content-Length, Content-Range, X-Content-Duration, X-Cache, X-Varnish

Curl seems to be able to download the files just fine.

Oct 20 2017, 4:53 PM · Patch-For-Review, media-storage, Operations, Traffic, Pywikibot-Commons, PAWS
zhuyifei1999 added a comment to T178567: Server error (500) while trying to download files from Commons from PAWS.

The same script runs perfectly fine on toolforge. I went ahead and checked the DNS resolving with host -v, both point to 208.80.154.240, upload-lb.eqiad.wikimedia.org.

Oct 20 2017, 4:32 PM · Patch-For-Review, media-storage, Operations, Traffic, Pywikibot-Commons, PAWS
zhuyifei1999 added a comment to T178567: Server error (500) while trying to download files from Commons from PAWS.

There are some headers however:

>>> req.data.headers
{'Date': 'Fri, 20 Oct 2017 16:19:09 GMT', 'Content-Type': 'text/plain', 'Content-Length': '37', 'Connection': 'keep-alive', 'X-Trans-Id':
'tx1c2bb6caeaeb431fb8433-0059ea21fd', 'Content-Encoding': 'gzip', 'Vary': 'Accept-Encoding', 'X-Varnish': '563519974, 332044385', 'Via': '
1.1 varnish-v4, 1.1 varnish-v4', 'Age': '0', 'X-Cache': 'cp1074 miss, cp1074 pass', 'X-Cache-Status': 'miss', 'Strict-Transport-Security':
 'max-age=106384710; includeSubDomains; preload', 'X-Analytics': 'https=1', 'X-Client-IP': '10.68.22.24', 'Access-Control-Allow-Origin': '
*', 'Access-Control-Expose-Headers': 'Age, Date, Content-Length, Content-Range, X-Content-Duration, X-Cache, X-Varnish', 'Timing-Allow-Ori
gin': '*'}
>>> for i in req.data.headers.items(): print('%s: %s' % i)
...
Date: Fri, 20 Oct 2017 16:19:09 GMT
Content-Type: text/plain
Content-Length: 37
Connection: keep-alive
X-Trans-Id: tx1c2bb6caeaeb431fb8433-0059ea21fd
Content-Encoding: gzip
Vary: Accept-Encoding
X-Varnish: 563519974, 332044385
Via: 1.1 varnish-v4, 1.1 varnish-v4
Age: 0
X-Cache: cp1074 miss, cp1074 pass
X-Cache-Status: miss
Strict-Transport-Security: max-age=106384710; includeSubDomains; preload
X-Analytics: https=1
X-Client-IP: 10.68.22.24
Access-Control-Allow-Origin: *
Access-Control-Expose-Headers: Age, Date, Content-Length, Content-Range, X-Content-Duration, X-Cache, X-Varnish
Timing-Allow-Origin: *
Oct 20 2017, 4:25 PM · Patch-For-Review, media-storage, Operations, Traffic, Pywikibot-Commons, PAWS
zhuyifei1999 added a comment to T178567: Server error (500) while trying to download files from Commons from PAWS.

I created a script that starts an interactive shell on error to debug this:

import pywikibot
from pywikibot import pagegenerators
Oct 20 2017, 4:21 PM · Patch-For-Review, media-storage, Operations, Traffic, Pywikibot-Commons, PAWS

Oct 19 2017

zhuyifei1999 removed a project from T178567: Server error (500) while trying to download files from Commons from PAWS: Pywikibot-General.
Oct 19 2017, 6:54 PM · Patch-For-Review, media-storage, Operations, Traffic, Pywikibot-Commons, PAWS
zhuyifei1999 added a comment to T178567: Server error (500) while trying to download files from Commons from PAWS.

Are the list of 500-ed images the same every time you execute the script? Or is the list random?

Oct 19 2017, 6:50 PM · Patch-For-Review, media-storage, Operations, Traffic, Pywikibot-Commons, PAWS
zhuyifei1999 added a comment to T178601: Add charset=utf-8 by default to lighttpd.

https://github.com/lighttpd/lighttpd2/blob/ca446911ef08e97417a74651ecd8528f9c7c27cf/contrib/create-mimetypes.conf.pl contains a more feature-rich create-mime.assign.pl that sets utf-8 in some conditions.

Oct 19 2017, 6:18 PM · Toolforge
zhuyifei1999 added a comment to T178601: Add charset=utf-8 by default to lighttpd.

The mimetype is assigned via: lighttpdwebservice.py#L44:

include_shell "/usr/share/lighttpd/create-mime.assign.pl"

which has these contents:

#06:01:51 0 ✓ zhuyifei1999@tools-webgrid-lighttpd-1401: ~$ cat /usr/share/lighttpd/create-mime.assign.pl
#!/usr/bin/perl -w
use strict;
open MIMETYPES, "/etc/mime.types" or exit;
print "mimetype.assign = (\n";
my %extensions;
while(<MIMETYPES>) {
  chomp;
  s/\#.*//;
  next if /^\w*$/;
  if(/^([a-z0-9\/+-.]+)\s+((?:[a-z0-9.+-]+[ ]?)+)$/) {
    foreach(split / /, $2) {
      # mime.types can have same extension for different
      # mime types
      next if $extensions{$_};
      $extensions{$_} = 1;
      print "\".$_\" => \"$1\",\n";
    }
  }
}
print ")\n";

Unfortunately, /etc/mime.types does not know whether a mime is binary or text:

06:05:54 0 ✓ zhuyifei1999@tools-bastion-02: ~$ grep -P 'javascript|css|html|txt|jpg|png|svg' /etc/mime.types
application/javascript				js
application/xhtml+xml				xhtml xht
application/vnd.ericsson.quickcall
application/vnd.pwg-xhtml-print+xml
#application/x-httpd-eruby			rhtml
#application/x-httpd-php			phtml pht php
image/jp2					jp2 jpg2
image/jpeg					jpeg jpg jpe
image/png					png
image/svg+xml					svg svgz
text/css					css
text/html					html htm shtml
text/plain					asc txt text pot brf srt
text/x-server-parsed-html

If we make charset=utf-8, we should preferably not affect the binary files.

Oct 19 2017, 6:10 PM · Toolforge

Oct 18 2017

zhuyifei1999 added a comment to T178519: Make backups of Quarry's main database on quarry-main-01.

I'm wondering, would it make sense for a separate instance to connect to quarry-main-01 to do the backups to its instance local storage? Or the backup process should be done on quarry-main-01 and rsync-ed to the separate instance?

Oct 18 2017, 6:36 PM · Quarry
zhuyifei1999 added a comment to T178459: Long running queries from pltools unlikely to finish.

I can not use EXPLAIN:

Oct 18 2017, 6:05 PM · Tools
zhuyifei1999 added a comment to T178520: Find somewhere else (not NFS) to store Quarry's resultsets.

The query runners must somehow store the results to somewhere the web server can access. Celery does not support sending large tables as a result of a job; doing so would flood the redis server, forever, as the query results are expected to be kept indefinitely.

Oct 18 2017, 6:03 PM · Quarry
zhuyifei1999 created T178520: Find somewhere else (not NFS) to store Quarry's resultsets.
Oct 18 2017, 6:00 PM · Quarry
zhuyifei1999 created T178519: Make backups of Quarry's main database on quarry-main-01.
Oct 18 2017, 5:53 PM · Quarry
zhuyifei1999 added a project to T178314: Transcodes has a very small max walltime limit of 3 minutes: Wikimedia-Video.
Oct 18 2017, 2:09 PM · Wikimedia-Video, MW-1.31-release-notes (WMF-deploy-2017-10-10 (1.31.0-wmf.3)), MW-1.30-release-notes, Patch-For-Review, MediaWiki-General-or-Unknown, Commons, Regression, TimedMediaHandler-Transcode

Oct 17 2017

zhuyifei1999 added a comment to T178314: Transcodes has a very small max walltime limit of 3 minutes.

@zhuyifei1999: Thanks a lot for your efforts to investigate this theme! I don't unterstand the technical details of your investigations.
Please, can you summarize the results of your activities? Did you detect a bug or a wrong configuration? What does it mean: Can the problem be solved by changing a configuration setting?

Oct 17 2017, 1:03 AM · Wikimedia-Video, MW-1.31-release-notes (WMF-deploy-2017-10-10 (1.31.0-wmf.3)), MW-1.30-release-notes, Patch-For-Review, MediaWiki-General-or-Unknown, Commons, Regression, TimedMediaHandler-Transcode

Oct 16 2017

zhuyifei1999 renamed T178314: Transcodes has a very small max walltime limit of 3 minutes from 4 webms (transcoded from a single mp4 with different methods) cannot be scaled on videoscalers, exitcode 124 to Transcodes has a very small max walltime limit of 3 minutes.
Oct 16 2017, 9:13 PM · Wikimedia-Video, MW-1.31-release-notes (WMF-deploy-2017-10-10 (1.31.0-wmf.3)), MW-1.30-release-notes, Patch-For-Review, MediaWiki-General-or-Unknown, Commons, Regression, TimedMediaHandler-Transcode
zhuyifei1999 added a project to T178314: Transcodes has a very small max walltime limit of 3 minutes: Commons.
Oct 16 2017, 9:10 PM · Wikimedia-Video, MW-1.31-release-notes (WMF-deploy-2017-10-10 (1.31.0-wmf.3)), MW-1.30-release-notes, Patch-For-Review, MediaWiki-General-or-Unknown, Commons, Regression, TimedMediaHandler-Transcode
zhuyifei1999 added projects to T178314: Transcodes has a very small max walltime limit of 3 minutes: Regression, MediaWiki-General-or-Unknown.

1bb9a22 by @MaxSem changed the fallback behavior from actually falling back to the minimum.

Oct 16 2017, 8:16 PM · Wikimedia-Video, MW-1.31-release-notes (WMF-deploy-2017-10-10 (1.31.0-wmf.3)), MW-1.30-release-notes, Patch-For-Review, MediaWiki-General-or-Unknown, Commons, Regression, TimedMediaHandler-Transcode
zhuyifei1999 added a comment to T178314: Transcodes has a very small max walltime limit of 3 minutes.

Adding the defined / expected ulimits (cpu time, mem, filesize) (1 2 3 4) did not reproduce the issue:

1​ zhuyifei1999@encoding01  ~  ulimit -f $(( 3 * 1024 * 1024 )); ulimit -v $(( 4 * 1024 * 1024 )); ulimit -t $(( 16 * 3600 )); time /usr/bin/ffmpeg -y -i /tmp/Vladimir_Putin_at_19th_World_Festival_of_Youth_and_Students_opening_ceremony.webm -threads 2 -skip_threshold 0 -bufsize 6000k -rc_init_occupancy 4000 -qmin 1 -qmax 51 -vb '2048000' -vcodec libvpx -slices '4' -f webm -s 1280x720 -aq '3' -acodec libvorbis /tmp/testout.webm
2​ffmpeg version 3.0.2-2~bpo8+1 Copyright (c) 2000-2016 the FFmpeg developers
3​ built with gcc 4.9.2 (Debian 4.9.2-10)
4​ configuration: --prefix=/usr --extra-version='2~bpo8+1' --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --cc=cc --cxx=g++ --enable-gpl --enable-shared --disable-stripping --disable-decoder=libopenjpeg --disable-decoder=libschroedinger --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-librubberband --enable-librtmp --enable-libschroedinger --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxvid --enable-libzvbi --enable-openal --enable-opengl --enable-x11grab --enable-libdc1394 --enable-libiec61883 --enable-libzmq --enable-frei0r --enable-chromaprint --enable-libx264
5​ libavutil 55. 17.103 / 55. 17.103
6​ libavcodec 57. 24.102 / 57. 24.102
7​ libavformat 57. 25.100 / 57. 25.100
8​ libavdevice 57. 0.101 / 57. 0.101
9​ libavfilter 6. 31.100 / 6. 31.100
10​ libavresample 3. 0. 0 / 3. 0. 0
11​ libswscale 4. 0.100 / 4. 0.100
12​ libswresample 2. 0.101 / 2. 0.101
13​ libpostproc 54. 0.100 / 54. 0.100
14​Input #0, matroska,webm, from '/tmp/Vladimir_Putin_at_19th_World_Festival_of_Youth_and_Students_opening_ceremony.webm':
15​ Metadata:
16​ encoder : Lavf57.25.100
17​ Duration: 00:04:43.08, start: 0.000000, bitrate: 3341 kb/s
18​ Stream #0:0(eng): Video: vp8, yuv420p, 1280x720, SAR 1:1 DAR 16:9, 25 fps, 25 tbr, 1k tbn, 1k tbc (default)
19​ Stream #0:1(eng): Audio: vorbis, 48000 Hz, stereo, fltp (default)
20​[libvpx @ 0x11f56a0] v1.3.0
21​Output #0, webm, to '/tmp/testout.webm':
22​ Metadata:
23​ encoder : Lavf57.25.100
24​ Stream #0:0(eng): Video: vp8 (libvpx), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], q=1-51, 2048 kb/s, 25 fps, 1k tbn, 25 tbc (default)
25​ Metadata:
26​ encoder : Lavc57.24.102 libvpx
27​ Side data:
28​ unknown side data type 10 (24 bytes)
29​ Stream #0:1(eng): Audio: vorbis (libvorbis), 48000 Hz, stereo, fltp (default)
30​ Metadata:
31​ encoder : Lavc57.24.102 libvorbis
32​Stream mapping:
33​ Stream #0:0 -> #0:0 (vp8 (native) -> vp8 (libvpx))
34​ Stream #0:1 -> #0:1 (vorbis (native) -> vorbis (libvorbis))
35​Press [q] to stop, [?] for help
36​frame= 7075 fps= 10 q=0.0 Lsize= 74934kB time=00:04:43.08 bitrate=2168.5kbits/s speed=0.415x
37​video:72430kB audio:2332kB subtitle:0kB other streams:0kB global headers:4kB muxing overhead: 0.229108%
38​/usr/bin/ffmpeg -y -i -threads 2 -skip_threshold 0 -bufsize 6000k 4000 -qmi 1223.49s user 79.56s system 191% cpu 11:22.10 total

Oct 16 2017, 7:59 PM · Wikimedia-Video, MW-1.31-release-notes (WMF-deploy-2017-10-10 (1.31.0-wmf.3)), MW-1.30-release-notes, Patch-For-Review, MediaWiki-General-or-Unknown, Commons, Regression, TimedMediaHandler-Transcode
zhuyifei1999 created P6131 T178314 Vladimir_Putin_at_19th_World_Festival_of_Youth_and_Students_opening_ceremony.webm.720p.webm encoding01 ulimit walltime.
Oct 16 2017, 7:42 PM