Page MenuHomePhabricator

zhuyifei1999
*Not* Serious business title.

Today

  • Clear sailing ahead.

Tomorrow

  • Clear sailing ahead.

Friday

  • Clear sailing ahead.

User Details

User Since
Oct 13 2014, 10:19 AM (249 w, 1 d)
Availability
Available
IRC Nick
zhuyifei1999
LDAP User
Zhuyifei1999
MediaWiki User
Zhuyifei1999 [ Global Accounts ]

There is currently no text in this page. You can search for this page title in other pages, or search the related logs.

Recent Activity

Thu, Jul 18

zhuyifei1999 committed rTVTCee67462bee52: download: Use empty UA for YouTube downloading (authored by zhuyifei1999).
download: Use empty UA for YouTube downloading
Thu, Jul 18, 6:35 AM

Tue, Jul 9

zhuyifei1999 added a comment to T166949: Homedir/UID info breaks after a while in Tools Kubernetes (can't read replica.my.cnf).

This is supposed to be set in the container spec: https://github.com/wikimedia/operations-software-tools-webservice/blob/master/toollabs/webservice/backends/kubernetesbackend.py#L438

Tue, Jul 9, 11:06 PM · Patch-For-Review, Tool-Global-user-contributions, cloud-services-team (Kanban), Kubernetes, Toolforge, Cloud-VPS

Mon, Jul 1

zhuyifei1999 edited projects for T226949: Query in Toolforge couldn't complete (timeout), added: Data-Services; removed Quarry.
Mon, Jul 1, 5:57 PM · Data-Services, Regression

Sun, Jun 30

zhuyifei1999 added a comment to T223052: Server-side upload request for Hurtigruten minutt for minutt videos.

Created T226937.

Sun, Jun 30, 7:31 PM · User-Urbanecm, Commons, Wikimedia-Site-requests

Wed, Jun 26

zhuyifei1999 added a comment to T226507: Port pywikibot-core to new pywikibot IV repository (tracking).

I don't remember what was the proposed solution at the meetup (maybe branch or tags, but no memory of creating a new repo at all), but the problem was discussed AFAICR. You also have to account for that at the meetup there are many of those who are just 'using pywikibot', but here, pretty much everyone is a pywikibot dev, who may have different priorities than end users.

Wed, Jun 26, 7:05 AM · Pywikibot-RfCs, Pywikibot

Tue, Jun 25

zhuyifei1999 added a comment to T226507: Port pywikibot-core to new pywikibot IV repository (tracking).

pwbc... pywikibot core; pwbd... pywikibot... daemon?

Tue, Jun 25, 5:56 PM · Pywikibot-RfCs, Pywikibot

Jun 19 2019

zhuyifei1999 added a comment to T225966: Toolforge: File upload to PHP webservice fails because temp file can't be opened.
[pid 27118] write(6, "172.16.6.39 tools.wmflabs.org - [19/Jun/2019:18:02:30 +0000] \"POST /wikiportretdev/upload.php HTTP/1.1\" 302 7506 \"https://tools."..., 248) = 248
Jun 19 2019, 6:21 PM · Toolforge
zhuyifei1999 added a comment to T225966: Toolforge: File upload to PHP webservice fails because temp file can't be opened.

Hmm, that flood of read() syscalls flooded my scrollback (I'm using screen). Would you try a smaller file? This is what it ended with (the file never reached PHP it seems):

373fV-\214\256\265\254!MR@\217\333\234\232O\227YOkk\7\273\357\216_Ll\n\300\232;b\310-"..., 4096) = 3000
[pid 27118] read(8, "", 4096)           = 0
[pid 27118] close(8)                    = 0
[pid 27118] accept4(4, 0x7ffcbd0539f0, [112], SOCK_CLOEXEC|SOCK_NONBLOCK) = -1 EAGAIN (Resource temporarily unavailable)
[pid 27118] open("/proc/loadavg", O_RDONLY) = 8
[pid 27118] read(8, "0.00 0.00 0.01 2/206 26797\n", 64) = 27
Jun 19 2019, 5:55 PM · Toolforge
zhuyifei1999 added a comment to T225966: Toolforge: File upload to PHP webservice fails because temp file can't be opened.

Did not get anything on the 4 PHPs. I just attached to lighttpd as well. Mind doing it again? Just to confirm, the tool is wikiportretdev right?

Jun 19 2019, 5:46 PM · Toolforge
zhuyifei1999 renamed T225966: Toolforge: File upload to PHP webservice fails because temp file can't be opened from WMFLabs: File upload fails because temp file can't be opened to Toolforge: File upload to PHP webservice fails because temp file can't be opened.
Jun 19 2019, 5:29 PM · Toolforge
zhuyifei1999 added a comment to T225966: Toolforge: File upload to PHP webservice fails because temp file can't be opened.

Just did some debugging and the issue already occurs on upload (just normal file upload via form POST). I tried to var_dump($_FILES);die(); and it already gives a Internal Server error before I even do anything else with the file.
All the error log says is:
2019-06-19 16:57:51: (chunk.c.553) opening temp-file failed: No such file or directory
2019-06-19 16:58:42: (chunk.c.553) opening temp-file failed: No such file or directory

Jun 19 2019, 5:26 PM · Toolforge
zhuyifei1999 added a comment to T225966: Toolforge: File upload to PHP webservice fails because temp file can't be opened.

What is the path that is being created / opened?

Jun 19 2019, 7:11 AM · Toolforge

Jun 7 2019

zhuyifei1999 committed rTVTC8ebed3fbe58a: urlextract: Remove emojies from titles (authored by zhuyifei1999).
urlextract: Remove emojies from titles
Jun 7 2019, 6:42 PM

May 31 2019

zhuyifei1999 added a comment to T223052: Server-side upload request for Hurtigruten minutt for minutt videos.

Had a delay in the progress because I didn't notice the during the scratch server migration the mount was set read-only.

May 31 2019, 10:36 PM · User-Urbanecm, Commons, Wikimedia-Site-requests
zhuyifei1999 updated the task description for T223052: Server-side upload request for Hurtigruten minutt for minutt videos.
May 31 2019, 10:35 PM · User-Urbanecm, Commons, Wikimedia-Site-requests
zhuyifei1999 updated subscribers of T153068: Consider mounting labs NFS labstore1003.eqiad.wmnet:/scratch for server-side uploads.

I don't mind which server it is mounting as long as it matches the current scratch server used in the video project. The point of this task was to make it easier to do server side uploads by removing the overhead of wget the files, but since @Dereckson hasn't been doing server-side uploads recently, I'd ask if @Reedy wants this.

May 31 2019, 10:28 PM · cloud-services-team (Kanban), Data-Services, Operations, video2commons

May 25 2019

zhuyifei1999 merged T224317: TSV variant of Quarry output is malformed into T224300: Extra tab is prepended to quoted fields in TSV output format.
May 25 2019, 12:20 PM · Patch-For-Review, Quarry
zhuyifei1999 merged task T224317: TSV variant of Quarry output is malformed into T224300: Extra tab is prepended to quoted fields in TSV output format.
May 25 2019, 12:20 PM · Quarry
zhuyifei1999 added a comment to T224317: TSV variant of Quarry output is malformed.

Sorry, T224300

May 25 2019, 12:20 PM · Quarry

May 24 2019

zhuyifei1999 added a comment to T224317: TSV variant of Quarry output is malformed.

Quotes are intended to the extent that any fields that contains a delimiter should be quoted. Extraneous tabs ("delimiters") are caused by T224317

May 24 2019, 10:39 PM · Quarry
zhuyifei1999 added a comment to T224317: TSV variant of Quarry output is malformed.

T224317?

May 24 2019, 9:06 PM · Quarry
zhuyifei1999 awarded T223620: Create a Toolforge tool to search wikitext across all wikis using Elastic Search a Love token.
May 24 2019, 7:45 PM · Tool-global-search, Wikimedia-Hackathon-2019
zhuyifei1999 added a comment to T224300: Extra tab is prepended to quoted fields in TSV output format.

This is from T209226: Quarry can be affected by CSV Injection. It's not supposed to hit every line. I'm looking into it.

May 24 2019, 5:35 PM · Patch-For-Review, Quarry
zhuyifei1999 updated the task description for T223052: Server-side upload request for Hurtigruten minutt for minutt videos.
May 24 2019, 1:37 AM · User-Urbanecm, Commons, Wikimedia-Site-requests

May 21 2019

zhuyifei1999 added a comment to T217501: Page banner of Wikivoyage can not automatically convert the problem of tranditional or simplified Chinese.

Wednesday, May 22 13:00–15:00 UTC

May 21 2019, 12:33 AM · Wikimedia-Hackathon-2019, MW-1.34-notes (1.34.0-wmf.6; 2019-05-21), Patch-For-Review, Language-Team, Readers-Web-Backlog (Tracking), Reading-Web-Local-Wiki-Issues, Wikidata-Page-Banner, Chinese-Sites

May 20 2019

zhuyifei1999 added a comment to T178520: Find somewhere else (not NFS) to store Quarry's resultsets.

Is the NFS server you're using being shared with other projects e.g. tools?

May 20 2019, 8:09 AM · Quarry

May 19 2019

zhuyifei1999 added a comment to T178520: Find somewhere else (not NFS) to store Quarry's resultsets.

It is a bottleneck only when people attempt to push our limits. T188564#4778741 has some graphs (the Wait I/O is D-sleep, NFS lag). I'm currently unable to reproduce so much NFS lag as in the graphs (maybe the situation with NFS has improved?)

May 19 2019, 11:56 PM · Quarry
zhuyifei1999 added a comment to T217501: Page banner of Wikivoyage can not automatically convert the problem of tranditional or simplified Chinese.

I'm fine either way

May 19 2019, 1:07 PM · Wikimedia-Hackathon-2019, MW-1.34-notes (1.34.0-wmf.6; 2019-05-21), Patch-For-Review, Language-Team, Readers-Web-Backlog (Tracking), Reading-Web-Local-Wiki-Issues, Wikidata-Page-Banner, Chinese-Sites
zhuyifei1999 added a comment to T223777: Add ca_ES.UTF-8 locale to Toolforge hosts.

is this because ca_ES.UTF-8 locale isn't installed?

May 19 2019, 12:57 PM · Toolforge (Software install/update)
zhuyifei1999 added a project to T217501: Page banner of Wikivoyage can not automatically convert the problem of tranditional or simplified Chinese: Wikimedia-Hackathon-2019.
May 19 2019, 12:54 PM · Wikimedia-Hackathon-2019, MW-1.34-notes (1.34.0-wmf.6; 2019-05-21), Patch-For-Review, Language-Team, Readers-Web-Backlog (Tracking), Reading-Web-Local-Wiki-Issues, Wikidata-Page-Banner, Chinese-Sites
zhuyifei1999 added a comment to T217501: Page banner of Wikivoyage can not automatically convert the problem of tranditional or simplified Chinese.

https://zh.wikivoyage.beta.wmflabs.org/zh-cn/%E6%B5%8B%E8%AF%95
https://zh.wikivoyage.beta.wmflabs.org/zh-hk/%E6%B5%8B%E8%AF%95

May 19 2019, 11:04 AM · Wikimedia-Hackathon-2019, MW-1.34-notes (1.34.0-wmf.6; 2019-05-21), Patch-For-Review, Language-Team, Readers-Web-Backlog (Tracking), Reading-Web-Local-Wiki-Issues, Wikidata-Page-Banner, Chinese-Sites
zhuyifei1999 added a comment to T223777: Add ca_ES.UTF-8 locale to Toolforge hosts.

Please post the output of your locale command.

May 19 2019, 10:10 AM · Toolforge (Software install/update)
Jdlrobson awarded T223770: Please create beta zhwikivoyage a Mountain of Wealth token.
May 19 2019, 8:09 AM · Chinese-Sites, Wikimedia-Hackathon-2019, Patch-For-Review, Beta-Cluster-Infrastructure
zhuyifei1999 added a comment to T217501: Page banner of Wikivoyage can not automatically convert the problem of tranditional or simplified Chinese.

T223770: Please create beta zhwikivoyage

May 19 2019, 8:05 AM · Wikimedia-Hackathon-2019, MW-1.34-notes (1.34.0-wmf.6; 2019-05-21), Patch-For-Review, Language-Team, Readers-Web-Backlog (Tracking), Reading-Web-Local-Wiki-Issues, Wikidata-Page-Banner, Chinese-Sites
zhuyifei1999 created T223770: Please create beta zhwikivoyage.
May 19 2019, 8:05 AM · Chinese-Sites, Wikimedia-Hackathon-2019, Patch-For-Review, Beta-Cluster-Infrastructure
zhuyifei1999 added a comment to T217501: Page banner of Wikivoyage can not automatically convert the problem of tranditional or simplified Chinese.

bd808 just gave me steward rights and still https://en.wikivoyage.beta.wmflabs.org/wiki/Special:PageLanguage doesn't exist. I guess it's configuration-disabled rather than lack of permissions. So yeah, let's create that wiki.

May 19 2019, 8:01 AM · Wikimedia-Hackathon-2019, MW-1.34-notes (1.34.0-wmf.6; 2019-05-21), Patch-For-Review, Language-Team, Readers-Web-Backlog (Tracking), Reading-Web-Local-Wiki-Issues, Wikidata-Page-Banner, Chinese-Sites

May 18 2019

zhuyifei1999 added a comment to T223750: Include pinyin for zhwiki damaging model.

These I don't use:

  • an, ang
  • l, n
  • f, h
  • l, r
  • g, k
May 18 2019, 8:22 PM · Chinese-Sites, editquality-modeling, artificial-intelligence, Scoring-platform-team
zhuyifei1999 updated the task description for T223052: Server-side upload request for Hurtigruten minutt for minutt videos.
May 18 2019, 6:53 PM · User-Urbanecm, Commons, Wikimedia-Site-requests
zhuyifei1999 updated the task description for T223052: Server-side upload request for Hurtigruten minutt for minutt videos.
May 18 2019, 6:51 PM · User-Urbanecm, Commons, Wikimedia-Site-requests
zhuyifei1999 added a comment to T221801: Pywikibot meetup (usage and/or development workshop, issues and future discussion, other stuff to talk about).

k. I'll be there

May 18 2019, 7:38 AM · Pywikibot, Wikimedia-Hackathon-2019
zhuyifei1999 added a comment to T221801: Pywikibot meetup (usage and/or development workshop, issues and future discussion, other stuff to talk about).

You mean 3PM? Sure. Where though?

May 18 2019, 7:28 AM · Pywikibot, Wikimedia-Hackathon-2019

May 17 2019

zhuyifei1999 added a comment to T217501: Page banner of Wikivoyage can not automatically convert the problem of tranditional or simplified Chinese.

You can try testing this out on https://en.wikivoyage.beta.wmflabs.org/?uselang=zh !

May 17 2019, 9:17 PM · Wikimedia-Hackathon-2019, MW-1.34-notes (1.34.0-wmf.6; 2019-05-21), Patch-For-Review, Language-Team, Readers-Web-Backlog (Tracking), Reading-Web-Local-Wiki-Issues, Wikidata-Page-Banner, Chinese-Sites
zhuyifei1999 closed T220272: Adoption request for Templatetransclusioncheck as Resolved.

Done. @Adithyak1997 I highly recommend you find a co-maintainer for this tool.

May 17 2019, 5:50 PM · Toolforge-standards-committee
zhuyifei1999 added a comment to T220272: Adoption request for Templatetransclusioncheck.

Does that mean the tool was run externally by just using db?

May 17 2019, 5:46 PM · Toolforge-standards-committee
zhuyifei1999 added a comment to T220272: Adoption request for Templatetransclusioncheck.

Tool does not own any databases:

tools.templatetransclusioncheck@tools-sgebastion-08:~/public_html$ id
uid=52410(tools.templatetransclusioncheck) gid=52410(tools.templatetransclusioncheck) groups=52410(tools.templatetransclusioncheck)
tools.templatetransclusioncheck@tools-sgebastion-08:~/public_html$ sql local
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 113317962
Server version: 10.1.38-MariaDB MariaDB Server
May 17 2019, 5:42 PM · Toolforge-standards-committee
zhuyifei1999 added a comment to T220272: Adoption request for Templatetransclusioncheck.

./.viminfo -- No secret data found
./replica.my.cnf -- Toolforge infra
./.cache -- contains single empty file .cache/motd.legal-displayed
./.ssh -- contains single file known_hosts
./.ssh/known_hosts -- Even if this contains secret data, it is hashed
./crontab.trusty.save -- No secret data found
./.kube/config -- Toolforge infra
./.bash_history -- No secret data found
./.lesshst -- No secret data found

May 17 2019, 5:38 PM · Toolforge-standards-committee
zhuyifei1999 claimed T220272: Adoption request for Templatetransclusioncheck.

(We talked abut this at the hackathon; going ahead)

May 17 2019, 5:19 PM · Toolforge-standards-committee
zhuyifei1999 added a comment to T217501: Page banner of Wikivoyage can not automatically convert the problem of tranditional or simplified Chinese.

Let's see if it works correctly next week when it's deployed. (I only tested in vagrant)

May 17 2019, 4:57 PM · Wikimedia-Hackathon-2019, MW-1.34-notes (1.34.0-wmf.6; 2019-05-21), Patch-For-Review, Language-Team, Readers-Web-Backlog (Tracking), Reading-Web-Local-Wiki-Issues, Wikidata-Page-Banner, Chinese-Sites
zhuyifei1999 awarded T221293: PAWS - How it works, what can it do, where can we take it? a Like token.
May 17 2019, 4:47 PM · PAWS, Wikimedia-Hackathon-2019
zhuyifei1999 claimed T217501: Page banner of Wikivoyage can not automatically convert the problem of tranditional or simplified Chinese.
May 17 2019, 2:02 PM · Wikimedia-Hackathon-2019, MW-1.34-notes (1.34.0-wmf.6; 2019-05-21), Patch-For-Review, Language-Team, Readers-Web-Backlog (Tracking), Reading-Web-Local-Wiki-Issues, Wikidata-Page-Banner, Chinese-Sites
zhuyifei1999 updated subscribers of T221657: 500 error when deleting ssh key from toolsadmin.
May 17 2019, 12:38 PM · cloud-services-team (Kanban), Striker

May 16 2019

zhuyifei1999 added a comment to T184712: Page.title(as_url=True) should encode also slashes.

No, the title was 'fixed', but is reopened for a (partial) revert afaict.

May 16 2019, 8:00 PM · Patch-For-Review, Pywikibot
zhuyifei1999 added a comment to T221801: Pywikibot meetup (usage and/or development workshop, issues and future discussion, other stuff to talk about).

Anyone at the bar?

May 16 2019, 6:21 PM · Pywikibot, Wikimedia-Hackathon-2019

May 15 2019

zhuyifei1999 updated the task description for T223052: Server-side upload request for Hurtigruten minutt for minutt videos.
May 15 2019, 3:56 AM · User-Urbanecm, Commons, Wikimedia-Site-requests
zhuyifei1999 renamed T223052: Server-side upload request for Hurtigruten minutt for minutt videos from Server-side upload request for Zhuyifei1999 to Server-side upload request for Hurtigruten minutt for minutt videos.
May 15 2019, 3:53 AM · User-Urbanecm, Commons, Wikimedia-Site-requests

May 13 2019

zhuyifei1999 created T223067: sudo is still broken on certain toolforge hosts.
May 13 2019, 8:23 AM · cloud-services-team (Kanban), Toolforge

May 12 2019

zhuyifei1999 added a project to T223052: Server-side upload request for Hurtigruten minutt for minutt videos: Commons.
May 12 2019, 11:01 PM · User-Urbanecm, Commons, Wikimedia-Site-requests
zhuyifei1999 created T223052: Server-side upload request for Hurtigruten minutt for minutt videos.
May 12 2019, 11:01 PM · User-Urbanecm, Commons, Wikimedia-Site-requests

May 11 2019

zhuyifei1999 added a comment to T192733: Remove old symlinks to trunk/rewrite/compat/pywikipedia in /shared.

Some people perhaps add it to PATH in their .profile etc.

May 11 2019, 11:49 PM · cloud-services-team (Kanban), Pywikibot, Toolforge
zhuyifei1999 added a comment to T192733: Remove old symlinks to trunk/rewrite/compat/pywikipedia in /shared.

If we really want to move forward with this task, how about this?

May 11 2019, 11:19 PM · cloud-services-team (Kanban), Pywikibot, Toolforge
zhuyifei1999 awarded T209119: Create a beta host a Love token.
May 11 2019, 11:03 PM · Quarry
zhuyifei1999 added a comment to T209119: Create a beta host.

Awesome thanks!

May 11 2019, 11:03 PM · Quarry

May 9 2019

zhuyifei1999 added a comment to T222255: tools-sgecron-01 virtual memory allocation error at midnight and noon UTC.

The reason given there is that supposedly jsub checks everything before submitting a job

May 9 2019, 4:47 AM · Patch-For-Review, cloud-services-team (Kanban), Toolforge

May 7 2019

zhuyifei1999 added a comment to T222255: tools-sgecron-01 virtual memory allocation error at midnight and noon UTC.

I'm curious about what is happening here that is new, though

May 7 2019, 3:48 AM · Patch-For-Review, cloud-services-team (Kanban), Toolforge
zhuyifei1999 added a comment to T222255: tools-sgecron-01 virtual memory allocation error at midnight and noon UTC.

There is a series of qstat processes segfaulting at 23:59:<seconds>

May 7 2019, 3:46 AM · Patch-For-Review, cloud-services-team (Kanban), Toolforge

May 6 2019

zhuyifei1999 added a comment to T222255: tools-sgecron-01 virtual memory allocation error at midnight and noon UTC.

There might be something useful in the grafana dashboards as well.

May 6 2019, 9:34 PM · Patch-For-Review, cloud-services-team (Kanban), Toolforge
zhuyifei1999 added a comment to T222255: tools-sgecron-01 virtual memory allocation error at midnight and noon UTC.

Why is memory overcommitting disabled? This config is ages old without explanation.

May 6 2019, 6:54 AM · Patch-For-Review, cloud-services-team (Kanban), Toolforge
zhuyifei1999 added a comment to T222255: tools-sgecron-01 virtual memory allocation error at midnight and noon UTC.

That is likely same issue; however, I don't think that has anything new in it.

May 6 2019, 6:21 AM · Patch-For-Review, cloud-services-team (Kanban), Toolforge
zhuyifei1999 added a comment to T222255: tools-sgecron-01 virtual memory allocation error at midnight and noon UTC.

As for the times:

$ sudo dmesg -H | grep segfault
[Apr 8 23:59] qstat[31987]: segfault at 0 ip 00007fb05483fa8e sp 00007fff050e8790 error 4 in libjemalloc.so.1[7fb05483b000+34000]
[Apr19 11:59] qstat[28885]: segfault at 0 ip 00007fe2da758a8e sp 00007ffcb2f5fd40 error 4 in libjemalloc.so.1[7fe2da754000+34000]
[  +0.012682] qstat[28904]: segfault at 0 ip 00007f5b4076ea8e sp 00007fff98dd21f0 error 4 in libjemalloc.so.1[7f5b4076a000+34000]
[  +0.003268] qstat[28940]: segfault at 0 ip 00007f7e8435ca8e sp 00007fffb91aa380 error 4 in libjemalloc.so.1[7f7e84358000+34000]
[  +0.012101] qstat[28900]: segfault at 0 ip 00007fb3c502da8e sp 00007ffcb824c980 error 4 in libjemalloc.so.1[7fb3c5029000+34000]
[Apr19 23:59] qstat[12949]: segfault at 0 ip 00007f68f3a97a8e sp 00007ffe062c2860 error 4 in libjemalloc.so.1[7f68f3a93000+34000]
[  +0.015150] sendmail[12972]: segfault at 7ffec3f72848 ip 0000565159ff90ad sp 00007ffec3f72850 error 6 in exim4[565159fc1000+ef000]
[  +0.052991] qstat[12955]: segfault at 0 ip 00007fe143e45a8e sp 00007ffd1493cd80 error 4 in libjemalloc.so.1[7fe143e41000+34000]
[Apr25 23:59] qstat[32482]: segfault at 0 ip 00007fb650160a8e sp 00007ffe8cb93210 error 4 in libjemalloc.so.1[7fb65015c000+34000]
[  +0.024774] qstat[32520]: segfault at 0 ip 00007f05a6f8ba8e sp 00007ffe951f1c40 error 4 in libjemalloc.so.1[7f05a6f87000+34000]
[  +0.017353] qstat[32564]: segfault at 0 ip 00007faebd9b3a8e sp 00007ffc3d4eaa10 error 4 in libjemalloc.so.1[7faebd9af000+34000]
[  +0.009400] qsub[32468]: segfault at 0 ip 00007fc67e3faa8e sp 00007ffc7c00e860 error 4 in libjemalloc.so.1[7fc67e3f6000+34000]
[  +0.053438] qstat[32348]: segfault at 0 ip 00007fd8ee58fa8e sp 00007ffc3dbaecb0 error 4 in libjemalloc.so.1[7fd8ee58b000+34000]
[  +0.033767] qstat[32287]: segfault at 0 ip 00007ff898412a8e sp 00007ffd3a0c0900 error 4 in libjemalloc.so.1[7ff89840e000+34000]
[  +0.984610] qstat[32687]: segfault at 0 ip 00007f3a83845a8e sp 00007ffd3cab3ca0 error 4 in libjemalloc.so.1[7f3a83841000+34000]
[  +0.000185] qstat[32669]: segfault at 0 ip 00007f7f4ff68a8e sp 00007ffcf0e9d8a0 error 4 in libjemalloc.so.1[7f7f4ff64000+34000]
[  +0.021948] qstat[32695]: segfault at 0 ip 00007faa956aaa8e sp 00007ffc4b4f0b00 error 4 in libjemalloc.so.1[7faa956a6000+34000]
[Apr27 23:59] qstat[31481]: segfault at 0 ip 00007f02593d0a8e sp 00007fff67cbb350 error 4 in libjemalloc.so.1[7f02593cc000+34000]
[  +0.001142] qstat[31490]: segfault at 0 ip 00007fdeb2837a8e sp 00007fff3f3979b0 error 4 in libjemalloc.so.1[7fdeb2833000+34000]
[  +0.208803] exim4[31704]: segfault at 8c ip 00007f048d0fb650 sp 00007ffc858c2b40 error 6 in libtasn1.so.6.5.3[7f048d0ef000+11000]
[  +0.096076] exim4[31653]: segfault at 8c ip 00007fce4ab6a650 sp 00007fffa615d900 error 6 in libtasn1.so.6.5.3[7fce4ab5e000+11000]
[  +0.850732] qstat[31516]: segfault at 0 ip 00007f6d01959a8e sp 00007fff28dc4570 error 4 in libjemalloc.so.1[7f6d01955000+34000]
[Apr30 23:59] qstat[21799]: segfault at 0 ip 00007fca45b51a8e sp 00007ffdcb0cc7e0 error 4 in libjemalloc.so.1[7fca45b4d000+34000]
[  +1.616116] qstat[22228]: segfault at 0 ip 00007f7f6f0d8a8e sp 00007ffdaed69aa0 error 4 in libjemalloc.so.1[7f7f6f0d4000+34000]
[  +0.039497] qstat[22236]: segfault at 0 ip 00007f89b3d23a8e sp 00007ffcd85e43f0 error 4 in libjemalloc.so.1[7f89b3d1f000+34000]
[May 2 23:59] qstat[31939]: segfault at 0 ip 00007f629ee35a8e sp 00007ffce25df370 error 4 in libjemalloc.so.1[7f629ee31000+34000]
[  +0.012823] qstat[31999]: segfault at 7ffd6ac12edc ip 00007f5e68cbbd4e sp 00007ffd6ac12e20 error 6 in libc-2.24.so[7f5e68c75000+195000]
[May 5 23:59] qstat[7476]: segfault at 0 ip 00007f47614f6a8e sp 00007ffcbe6a3340 error 4 in libjemalloc.so.1[7f47614f2000+34000]
[  +0.061838] qstat[7495]: segfault at 0 ip 00007fd66faefa8e sp 00007fff0bba5e50 error 4 in libjemalloc.so.1[7fd66faeb000+34000]
[  +0.030899] qstat[7502]: segfault at 0 ip 00007feb54518a8e sp 00007fff36b34c10 error 4 in libjemalloc.so.1[7feb54514000+34000]
[  +0.021800] qstat[7507]: segfault at 0 ip 00007fd483651a8e sp 00007fff47717cf0 error 4 in libjemalloc.so.1[7fd48364d000+34000]

Any time not here is highly unlikely to be related.

May 6 2019, 5:55 AM · Patch-For-Review, cloud-services-team (Kanban), Toolforge
zhuyifei1999 added a comment to T222255: tools-sgecron-01 virtual memory allocation error at midnight and noon UTC.
May 6 2019, 5:51 AM · Patch-For-Review, cloud-services-team (Kanban), Toolforge
zhuyifei1999 added a comment to T222255: tools-sgecron-01 virtual memory allocation error at midnight and noon UTC.

Quote T222429:

/usr/lib/gridengine/qstat: error while loading shared libraries: libm.so.6: failed to map segment from shared object
May 6 2019, 4:48 AM · Patch-For-Review, cloud-services-team (Kanban), Toolforge
zhuyifei1999 merged T222429: xml.etree.ElementTree.ParseError: no element found: line 1, column 0 on job submission through Cron into T222255: tools-sgecron-01 virtual memory allocation error at midnight and noon UTC.
May 6 2019, 4:41 AM · Patch-For-Review, cloud-services-team (Kanban), Toolforge
zhuyifei1999 merged task T222429: xml.etree.ElementTree.ParseError: no element found: line 1, column 0 on job submission through Cron into T222255: tools-sgecron-01 virtual memory allocation error at midnight and noon UTC.
May 6 2019, 4:41 AM · cloud-services-team (Kanban), Toolforge
zhuyifei1999 added a comment to T222255: tools-sgecron-01 virtual memory allocation error at midnight and noon UTC.

I got a different error at 2019-05-06T00:00. Not sure if it is related.

May 6 2019, 3:17 AM · Patch-For-Review, cloud-services-team (Kanban), Toolforge
zhuyifei1999 added a comment to T222255: tools-sgecron-01 virtual memory allocation error at midnight and noon UTC.

Missed my chance today to capture an strace of what is going on. Was using bashisms in my crontab :(

May 6 2019, 2:56 AM · Patch-For-Review, cloud-services-team (Kanban), Toolforge

May 5 2019

zhuyifei1999 raised the priority of T222255: tools-sgecron-01 virtual memory allocation error at midnight and noon UTC from Low to High.
May 5 2019, 6:20 PM · Patch-For-Review, cloud-services-team (Kanban), Toolforge
zhuyifei1999 added a comment to T222430: MemoryError on submitting a job through Toolforge's cron.

@Urbanecm Could you check whether the time in which the email was sent was 0 midnight UTC?

May 5 2019, 6:20 PM · cloud-services-team (Kanban), Toolforge
zhuyifei1999 renamed T222255: tools-sgecron-01 virtual memory allocation error at midnight and noon UTC from Cron gridengine jobs are failing for memory errors to tools-sgecron-01 virtual memory allocation error at midnight and noon UTC.
May 5 2019, 5:38 PM · Patch-For-Review, cloud-services-team (Kanban), Toolforge
zhuyifei1999 added a comment to T222255: tools-sgecron-01 virtual memory allocation error at midnight and noon UTC.

There are around 467 jobs that gets submitted at midnight UTC:

05:32:44 0 ✓ zhuyifei1999@tools-sgecron-01: ~$ sudo bash -c 'cat /var/spool/cron/crontabs/*' | sed '/^ *#/d' | sed '/^ *$/d' | sed -r 's/\s+/ /g' | sed -r 's/^ //' | awk '(($1 ~ /\*/ || $1 ~ /(^|,)0($|,)/) && ($2 ~ /\*/ || $2 ~ /(^|,)0($|,)/)) { print $0 }' | wc -l
467
May 5 2019, 5:34 PM · Patch-For-Review, cloud-services-team (Kanban), Toolforge
zhuyifei1999 added a comment to T221451: An error in Fireflytools.

Then it has nothing to do with k8s. Cron doesn't do k8s.

May 5 2019, 5:19 PM · Tools, Toolforge
zhuyifei1999 merged T221451: An error in Fireflytools into T222255: tools-sgecron-01 virtual memory allocation error at midnight and noon UTC.
May 5 2019, 5:18 PM · Patch-For-Review, cloud-services-team (Kanban), Toolforge
zhuyifei1999 merged task T221451: An error in Fireflytools into T222255: tools-sgecron-01 virtual memory allocation error at midnight and noon UTC.
May 5 2019, 5:18 PM · Tools, Toolforge
zhuyifei1999 added a comment to T221451: An error in Fireflytools.

@Adithyak1997 Who is the sender of the email you are referring to by 'I have got an email recently with this message.'?

May 5 2019, 5:15 PM · Tools, Toolforge
zhuyifei1999 added a comment to T221451: An error in Fireflytools.

This bug seems similar to T222255. I don't think it has anything to do with -mem

May 5 2019, 4:42 PM · Tools, Toolforge

May 1 2019

zhuyifei1999 added a comment to T222255: tools-sgecron-01 virtual memory allocation error at midnight and noon UTC.

There is no calls to brk() after libjemalloc is mapped into memory:

08:19:14 0 ✓ zhuyifei1999@tools-sgecron-01: ~$ strace -e brk,mmap,open,openat qstat
brk(NULL)                               = 0x5566d1885000
open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
mmap(NULL, 105645, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f4fcbcf7000
open("/lib/x86_64-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f4fcbcf5000
mmap(NULL, 3795296, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f4fcb74f000
mmap(0x7f4fcbae4000, 24576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x195000) = 0x7f4fcbae4000
mmap(0x7f4fcbaea000, 14688, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f4fcbaea000
brk(NULL)                               = 0x5566d1885000
brk(0x5566d18a6000)                     = 0x5566d18a6000
open("/usr/bin/qstat", O_RDONLY)        = 3
--- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=2863, si_uid=3306, si_status=0, si_utime=0, si_stime=0} ---
open("/etc/default/gridengine", O_RDONLY) = 3
brk(NULL)                               = 0x563f06955000
open("/usr/lib/gridengine/../../lib/lx-amd64/tls/x86_64/libmunge.so.2", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/usr/lib/gridengine/../../lib/lx-amd64/tls/libmunge.so.2", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/usr/lib/gridengine/../../lib/lx-amd64/x86_64/libmunge.so.2", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/usr/lib/gridengine/../../lib/lx-amd64/libmunge.so.2", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
mmap(NULL, 105645, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f05a9987000
open("/usr/lib/x86_64-linux-gnu/libmunge.so.2", O_RDONLY|O_CLOEXEC) = 3
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f05a9985000
mmap(NULL, 2135336, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f05a9574000
mmap(0x7f05a977c000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x8000) = 0x7f05a977c000
open("/usr/lib/x86_64-linux-gnu/libssl.so.1.0.2", O_RDONLY|O_CLOEXEC) = 3
mmap(NULL, 2526448, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f05a930b000
mmap(0x7f05a956a000, 40960, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x5f000) = 0x7f05a956a000
open("/usr/lib/x86_64-linux-gnu/libcrypto.so.1.0.2", O_RDONLY|O_CLOEXEC) = 3
mmap(NULL, 4609664, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f05a8ea5000
mmap(0x7f05a92dc000, 176128, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x237000) = 0x7f05a92dc000
mmap(0x7f05a9307000, 13952, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f05a9307000
open("/lib/x86_64-linux-gnu/libdl.so.2", O_RDONLY|O_CLOEXEC) = 3
mmap(NULL, 2109680, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f05a8ca1000
mmap(0x7f05a8ea3000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2000) = 0x7f05a8ea3000
open("/lib/x86_64-linux-gnu/libm.so.6", O_RDONLY|O_CLOEXEC) = 3
mmap(NULL, 3158248, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f05a899d000
mmap(0x7f05a8c9f000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x102000) = 0x7f05a8c9f000
open("/lib/x86_64-linux-gnu/libpthread.so.0", O_RDONLY|O_CLOEXEC) = 3
mmap(NULL, 2212936, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f05a8780000
mmap(0x7f05a8997000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x17000) = 0x7f05a8997000
mmap(0x7f05a8999000, 13384, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f05a8999000
open("/usr/lib/x86_64-linux-gnu/libjemalloc.so.1", O_RDONLY|O_CLOEXEC) = 3
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f05a9983000
mmap(NULL, 2319504, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f05a8549000
mmap(0x7f05a877c000, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x33000) = 0x7f05a877c000
mmap(0x7f05a877f000, 1168, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f05a877f000
open("/lib/x86_64-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
mmap(NULL, 3795296, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f05a81aa000
mmap(0x7f05a853f000, 24576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x195000) = 0x7f05a853f000
mmap(0x7f05a8545000, 14688, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f05a8545000
mmap(NULL, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f05a9980000
mmap(NULL, 4194304, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f05a7daa000
mmap(NULL, 8384512, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f05a79ab000
open("/sys/devices/system/cpu/online", O_RDONLY|O_CLOEXEC) = 3
mmap(NULL, 217032, PROT_READ, MAP_SHARED, 4, 0) = 0x7f05a994b000
mmap(NULL, 4194304, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f05a7800000
mmap(NULL, 102803208, PROT_READ, MAP_SHARED, 4, 0) = 0x7f05a15f5000
mmap(NULL, 102803208, PROT_READ, MAP_SHARED, 4, 0) = 0x7f059b3ea000
open("/etc/resolv.conf", O_RDONLY|O_CLOEXEC) = 3
mmap(NULL, 1010648, PROT_READ, MAP_SHARED, 4, 0) = 0x7f05a9854000
open("/var/lib/gridengine/default/common/bootstrap", O_RDONLY) = 3
open("/etc/resolv.conf", O_RDONLY|O_CLOEXEC) = 3
open("/var/lib/gridengine/default/common/act_qmaster", O_RDONLY) = 3
+++ exited with 0 +++

So, AFAICT, the kernel must have refused a call to mmap for some weird reason.

May 1 2019, 8:22 PM · Patch-For-Review, cloud-services-team (Kanban), Toolforge
zhuyifei1999 added a comment to T222255: tools-sgecron-01 virtual memory allocation error at midnight and noon UTC.

jsub does invoke job in is_running() function at line 281.

May 1 2019, 8:16 PM · Patch-For-Review, cloud-services-team (Kanban), Toolforge
zhuyifei1999 added a comment to T222255: tools-sgecron-01 virtual memory allocation error at midnight and noon UTC.

even sent me emails with the stacktrace/traceback

May 1 2019, 3:03 PM · Patch-For-Review, cloud-services-team (Kanban), Toolforge

Apr 30 2019

zhuyifei1999 committed rTVTCd61d4d797fd1: transcodejob.py: Make time limit wall clock time rather than CPU time (authored by zhuyifei1999).
transcodejob.py: Make time limit wall clock time rather than CPU time
Apr 30 2019, 7:30 PM

Apr 25 2019

zhuyifei1999 added a comment to T221801: Pywikibot meetup (usage and/or development workshop, issues and future discussion, other stuff to talk about).

solve sysop dualism

Apr 25 2019, 7:10 PM · Pywikibot, Wikimedia-Hackathon-2019

Apr 24 2019

zhuyifei1999 added a project to T217503: Tool to cut videos in Commons: Wikimedia-Video.
Apr 24 2019, 10:50 PM · Wikimedia-Video, Google-Summer-of-Code (2019), Technical-Tool-Request
zhuyifei1999 added a comment to T217503: Tool to cut videos in Commons.

If Toolforge's resource constraints impair the tool's availability and speed, I can offer https://wikitech.wikimedia.org/wiki/Nova_Resource:Video for the encoding part. This is what video2commons uses. A more updated ffmpeg with VP9 row-mt is also available there.

Apr 24 2019, 10:48 PM · Wikimedia-Video, Google-Summer-of-Code (2019), Technical-Tool-Request

Apr 11 2019

zhuyifei1999 added a comment to T220769: Account created without having a loginwiki or metawiki automatically created.

API creation didn't reproduce. Arguments:
https://commons.wikimedia.org/wiki/Special:ApiSandbox#action=createaccount&format=json&createreturnurl=https%3A%2F%2Fwww.example.com&createtoken=3c06ea9f6546cfd0a8ef9c2ef2a2568e5cafb7e1%2B%5C&username=Zhuyifei1999%20(T220769%20test)&password=REDACTED&retype=REDACTED&captchaId=256230518&captchaWord=birdstong

Apr 11 2019, 10:04 PM · Stewards-and-global-tools, MediaWiki-extensions-CentralAuth

Apr 5 2019

zhuyifei1999 closed T209226: Quarry can be affected by CSV Injection as Resolved.

Feel free to set this as public (I think).

Apr 5 2019, 6:50 PM · Quarry, Security
zhuyifei1999 added a comment to T209226: Quarry can be affected by CSV Injection.

https://tools.wmflabs.org/sal/quarry:

2019-04-05
18:48 	<zhuyifei1999_> 	checked out FETCH_HEAD on quarry-web-01 T209226
18:43 	<zhuyifei1999_> 	applied 0001-SECURITY-escape-CSV-injections.patch on quarry-web-01 and restarted uwsgi T209226
Apr 5 2019, 6:49 PM · Quarry, Security
zhuyifei1999 added a comment to T209226: Quarry can be affected by CSV Injection.

https://gerrit.wikimedia.org/r/#/c/analytics/quarry/web/+/501662/ Merged.

Apr 5 2019, 6:46 PM · Quarry, Security
zhuyifei1999 added a comment to T209226: Quarry can be affected by CSV Injection.
$ curl https://quarry.wmflabs.org/run/359561/output/0/csv
	=1+1
	=1+1
Apr 5 2019, 6:45 PM · Quarry, Security
zhuyifei1999 added a comment to T209226: Quarry can be affected by CSV Injection.

Alright, I'll deploy it after I finish lunch today.

Apr 5 2019, 6:09 PM · Quarry, Security

Mar 27 2019

zhuyifei1999 added a comment to T219351: Java jobs run the Stretch grid seem to require a very large memory reservation.

Oh and, gdb doesn't help on why it needs 10GiB of heap space.

Mar 27 2019, 3:01 AM · Toolforge
zhuyifei1999 added a comment to T219351: Java jobs run the Stretch grid seem to require a very large memory reservation.

It wants 10 GiB of heap space... but what's wrong with giving it just that? It's not like it's a consumable given that every process has their own address space.

Mar 27 2019, 2:59 AM · Toolforge
zhuyifei1999 created P8286 (An Untitled Masterwork).
Mar 27 2019, 2:56 AM