zhuyifei1999
*Not* Serious business title.

Projects (19)

Today

  • Clear sailing ahead.

Tomorrow

  • Clear sailing ahead.

Wednesday

  • Clear sailing ahead.

User Details

User Since
Oct 13 2014, 10:19 AM (214 w, 7 h)
Availability
Available
IRC Nick
zhuyifei1999
LDAP User
Zhuyifei1999
MediaWiki User
Zhuyifei1999 [ Global Accounts ]

There is currently no text in this page. You can search for this page title in other pages, or search the related logs.

Recent Activity

Yesterday

zhuyifei1999 added a comment to T209780: Toolforge module issue.

Then upgrade setuptools, as you are directed.

Sun, Nov 18, 7:11 PM · Toolforge
zhuyifei1999 added a comment to T209780: Toolforge module issue.

Have you looked into the link I gave you?

Sun, Nov 18, 6:50 PM · Toolforge
zhuyifei1999 added a comment to T209780: Toolforge module issue.

But I have installed toolforge through the windows command prompt by using the command "pip install toolforge".

Sun, Nov 18, 6:36 PM · Toolforge
zhuyifei1999 closed T209780: Toolforge module issue as Invalid.

When I accessed the file long_redirects.err, the last statement is shown below:

Sun, Nov 18, 6:23 PM · Toolforge
zhuyifei1999 updated the task description for T209780: Toolforge module issue.
Sun, Nov 18, 6:06 PM · Toolforge

Fri, Nov 16

zhuyifei1999 added a comment to T209147: Adoption request for Firefly Tools.

There are a lot of docs on wikitech: https://wikitech.wikimedia.org/wiki/Help:Toolforge

Fri, Nov 16, 7:20 AM · Toolforge-standards-committee
zhuyifei1999 added a comment to T209632: Move 'video' project to eqiad1-r.

encoding02 has been depooled.

Fri, Nov 16, 5:36 AM · Patch-For-Review, cloud-services-team (Kanban), Cloud-Services

Thu, Nov 15

zhuyifei1999 added a comment to T209632: Move 'video' project to eqiad1-r.

encoding03 is idle and not pooled:

zhuyifei1999@encoding01:~$ sudo celery -A video2commons.backend.worker --workdir=/srv/v2c status
celery1@encoding01: OK
celery2@encoding01: OK
celery1@encoding02: OK
celery2@encoding02: OK
Thu, Nov 15, 7:38 PM · Patch-For-Review, cloud-services-team (Kanban), Cloud-Services
zhuyifei1999 added a comment to T209147: Adoption request for Firefly Tools.

FWIW, this tool has a git repo at https://github.com/rwjuk/fireflytools, checked out at commit 61e219fe816fc44381f25576282e02fc334f3ab9, and

Thu, Nov 15, 6:40 PM · Toolforge-standards-committee
zhuyifei1999 closed T209557: VP9 in Video2Commons fails (due to Opus encoding broken) as Invalid.

It's an issue with Opus. https://github.com/toolforge/video2commons/issues/15 I'll see if I can do it this weekend.

Thu, Nov 15, 6:50 AM · video2commons

Wed, Nov 14

zhuyifei1999 added a comment to T209147: Adoption request for Firefly Tools.

This tool is python based so I think I can do the checking, but I have little time till next weekend (might have some time Thursday morning/noon). Toolforge-standards-committee Anyone else willing to do the checking?

Wed, Nov 14, 5:03 AM · Toolforge-standards-committee
zhuyifei1999 added a comment to T209147: Adoption request for Firefly Tools.
  • wikitech usertalk pages, per LINK(s)
Wed, Nov 14, 2:05 AM · Toolforge-standards-committee

Tue, Nov 13

zhuyifei1999 committed rTVTCa89a7e80582d: Revert "Smarter youtube-dl format selection" (authored by zhuyifei1999).
Revert "Smarter youtube-dl format selection"
Tue, Nov 13, 3:52 PM
zhuyifei1999 added a reverting change for rTVTC3c29ade4506a: Smarter youtube-dl format selection: rTVTCa89a7e80582d: Revert "Smarter youtube-dl format selection".
Tue, Nov 13, 3:52 PM

Mon, Nov 12

zhuyifei1999 committed rTVTC3c29ade4506a: Smarter youtube-dl format selection (authored by zhuyifei1999).
Smarter youtube-dl format selection
Mon, Nov 12, 8:32 PM

Sat, Nov 10

zhuyifei1999 added a comment to T209200: titleblacklist-forbidden when uploading to commons through pywikibot.

I can upload the very same file from Flickr using the built-in feature in Commons upload form and it accepts it

Sat, Nov 10, 6:41 AM · Pywikibot, Commons

Fri, Nov 9

zhuyifei1999 added a comment to T209119: Create a beta host.

I think the current problem is that the results are stored on NFS, and you can't possibly "namespace" NFS with beta and production.

Fri, Nov 9, 12:18 AM · Quarry

Thu, Nov 8

zhuyifei1999 changed the status of T185561: weblinkchecker.py slows down (itself, OS) to freeze after a while reaching 100% of RAM from Open to Stalled.
Thu, Nov 8, 11:34 PM · Patch-For-Review, Pywikibot-network, Pywikibot-weblinkchecker.py, Pywikibot
zhuyifei1999 added a comment to T185561: weblinkchecker.py slows down (itself, OS) to freeze after a while reaching 100% of RAM.

(still waiting for upstream release)

Thu, Nov 8, 8:49 PM · Patch-For-Review, Pywikibot-network, Pywikibot-weblinkchecker.py, Pywikibot
zhuyifei1999 updated subscribers of T209094: weblinkchecker.py: TypeError: 'unicode' object is not callable.

This is a Regression from 482a453. CC @D3r1ck01.

Thu, Nov 8, 8:00 PM · Regression, Patch-For-Review, Pywikibot
zhuyifei1999 renamed T209094: weblinkchecker.py: TypeError: 'unicode' object is not callable from Unicode traceback in weblinkchecker.py to weblinkchecker.py: TypeError: 'unicode' object is not callable.
Thu, Nov 8, 7:56 PM · Regression, Patch-For-Review, Pywikibot

Wed, Nov 7

zhuyifei1999 moved T209000: Update status when internal error occurs in worker from Backlog to Bugs (Endless Query Run) on the Quarry board.
Wed, Nov 7, 11:32 PM · Quarry
zhuyifei1999 added a comment to T209000: Update status when internal error occurs in worker.

So the question becomes: what happens when an error occurs while updating the status to 'failed' ;)

Wed, Nov 7, 11:27 PM · Quarry

Tue, Nov 6

zhuyifei1999 added a comment to T208865: tools-paws-master-01 SSH failure.

Yeah, my root key works now.

zhuyifei1999@zhuyifei1999-ThinkPad-X260:~$ ssh root@tools-paws-master-01.tools.eqiad.wmflabs
Linux tools-paws-master-01 4.9.0-6-amd64 #1 SMP Debian 4.9.88-1+deb9u1 (2018-05-07) x86_64
Debian GNU/Linux 9.5 (stretch)
tools-paws-master-01 is a server part of the Toolforge cluster (toollabs::base)
The last Puppet run was at Tue Nov  6 16:52:56 UTC 2018 (0 minutes ago). 
Last login: Sat Jun 30 16:42:54 2018 from bastion-01.bastion.eqiad.wmflabs
root@tools-paws-master-01:~# logout
Connection to tools-paws-master-01.tools.eqiad.wmflabs closed.
Tue, Nov 6, 4:54 PM · PAWS, cloud-services-team (Kanban)
zhuyifei1999 added a comment to T208865: tools-paws-master-01 SSH failure.

I don't see anything interesting in the IRC logs, SAL, or puppet commit messages that day, without really looking into the details. Let me see if changing the master fixes it.

Tue, Nov 6, 4:50 PM · PAWS, cloud-services-team (Kanban)
zhuyifei1999 added a comment to T208865: tools-paws-master-01 SSH failure.

The logs only go back to Oct 30 :(

Tue, Nov 6, 4:45 PM · PAWS, cloud-services-team (Kanban)
zhuyifei1999 added a comment to T208865: tools-paws-master-01 SSH failure.

The modification of puppet.conf corresponds well with the time of last successful puppet run:

04:39:20 0 ✓ zhuyifei1999@tools-paws-master-01: ~$ stat /etc/puppet/puppet.conf
  File: /etc/puppet/puppet.conf
  Size: 593       	Blocks: 8          IO Block: 4096   regular file
Device: fe03h/65027d	Inode: 655841      Links: 1
Access: (0444/-r--r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2018-11-06 10:33:03.570716470 +0000
Modify: 2018-10-26 08:32:40.473636506 +0000
Change: 2018-10-26 08:32:40.473636506 +0000
 Birth: -
Tue, Nov 6, 4:41 PM · PAWS, cloud-services-team (Kanban)
zhuyifei1999 added a comment to T208865: tools-paws-master-01 SSH failure.

root key doesn't work but user key works:

zhuyifei1999@zhuyifei1999-ThinkPad-X260:~$ ssh root@tools-paws-master-01.tools.eqiad.wmflabs
root@tools-paws-master-01.tools.eqiad.wmflabs: Permission denied (publickey).
zhuyifei1999@zhuyifei1999-ThinkPad-X260:~$ ssh tools-paws-master-01.tools.eqiad.wmflabs
Linux tools-paws-master-01 4.9.0-6-amd64 #1 SMP Debian 4.9.88-1+deb9u1 (2018-05-07) x86_64
Debian GNU/Linux 9.5 (stretch)
tools-paws-master-01 is a server part of the Toolforge cluster (toollabs::base)
The last Puppet run was at Fri Oct 26 08:32:33 UTC 2018 (16297 minutes ago). 
04:10:34 0 ✓ zhuyifei1999@tools-paws-master-01: ~$

This puppet really old. Shall I force a puppet run?

Tue, Nov 6, 4:11 PM · PAWS, cloud-services-team (Kanban)
zhuyifei1999 added a comment to T208855: tools: Hosts with local non-admin crontabs.

It's prometheus (a system user):

root@tools-webgrid-lighttpd-1428:~# ls /var/spool/cron/crontabs/
prometheus  root
root@tools-webgrid-lighttpd-1428:~# cat /var/spool/cron/crontabs/prometheus 
# DO NOT EDIT THIS FILE - edit the master and reinstall.
# (- installed on Wed Jul  5 11:40:49 2017)
# (Cron version -- $Id: crontab.c,v 2.13 1994/01/17 03:20:37 vixie Exp $)
# HEADER: This file was autogenerated at 2017-07-05 11:40:49 +0000 by puppet.
# HEADER: While it can still be managed manually, it is definitely not recommended.
# HEADER: Note particularly that the comments starting with 'Puppet Name' should
# HEADER: not be deleted, as doing so could cause duplicate cron jobs.
# Puppet Name: prometheus_puppet_agent_stats
* * * * * /usr/local/bin/prometheus-puppet-agent-stats --outfile /var/lib/prometheus/node.d/puppet_agent.prom

We should perhaps whitelist it.

Tue, Nov 6, 3:36 PM · Patch-For-Review, Toolforge, cloud-services-team (Kanban)

Mon, Nov 5

zhuyifei1999 closed T207677: Migrate 'Quarry' project to eqiad1 as Resolved.
Mon, Nov 5, 6:38 PM · Patch-For-Review, Quarry, cloud-services-team (Kanban), Cloud-Services
zhuyifei1999 closed T207677: Migrate 'Quarry' project to eqiad1, a subtask of T204745: cloudvps: migrate projects from main to eqiad1, as Resolved.
Mon, Nov 5, 6:38 PM · cloud-services-team (Kanban), Patch-For-Review, Cloud-Services
zhuyifei1999 added a comment to T207677: Migrate 'Quarry' project to eqiad1.

This happened after restore:

Nov 05 18:25:17 quarry-web-01 uwsgi-quarry-web[1815]: [2018-11-05 18:25:17,158] ERROR in app: Exception on /api/query/run [POST]
Nov 05 18:25:17 quarry-web-01 uwsgi-quarry-web[1815]: Traceback (most recent call last):
Nov 05 18:25:17 quarry-web-01 uwsgi-quarry-web[1815]:   File "/srv/quarry/venv/lib/python3.5/site-packages/flask/app.py", line 2292, in wsgi_app
Nov 05 18:25:17 quarry-web-01 uwsgi-quarry-web[1815]:     response = self.full_dispatch_request()
Nov 05 18:25:17 quarry-web-01 uwsgi-quarry-web[1815]:   File "/srv/quarry/venv/lib/python3.5/site-packages/flask/app.py", line 1815, in full_dispatch_request
Nov 05 18:25:17 quarry-web-01 uwsgi-quarry-web[1815]:     rv = self.handle_user_exception(e)
Nov 05 18:25:17 quarry-web-01 uwsgi-quarry-web[1815]:   File "/srv/quarry/venv/lib/python3.5/site-packages/flask/app.py", line 1718, in handle_user_exception
Nov 05 18:25:17 quarry-web-01 uwsgi-quarry-web[1815]:     reraise(exc_type, exc_value, tb)
Nov 05 18:25:17 quarry-web-01 uwsgi-quarry-web[1815]:   File "/srv/quarry/venv/lib/python3.5/site-packages/flask/_compat.py", line 35, in reraise
Nov 05 18:25:17 quarry-web-01 uwsgi-quarry-web[1815]:     raise value
Nov 05 18:25:17 quarry-web-01 uwsgi-quarry-web[1815]:   File "/srv/quarry/venv/lib/python3.5/site-packages/flask/app.py", line 1813, in full_dispatch_request
Nov 05 18:25:17 quarry-web-01 uwsgi-quarry-web[1815]:     rv = self.dispatch_request()
Nov 05 18:25:17 quarry-web-01 uwsgi-quarry-web[1815]:   File "/srv/quarry/venv/lib/python3.5/site-packages/flask/app.py", line 1799, in dispatch_request
Nov 05 18:25:17 quarry-web-01 uwsgi-quarry-web[1815]:     return self.view_functions[rule.endpoint](**req.view_args)
Nov 05 18:25:17 quarry-web-01 uwsgi-quarry-web[1815]:   File "./quarry/web/app.py", line 236, in api_run_query
Nov 05 18:25:17 quarry-web-01 uwsgi-quarry-web[1815]:     if not result.ready():
Nov 05 18:25:17 quarry-web-01 uwsgi-quarry-web[1815]:   File "/srv/quarry/venv/lib/python3.5/site-packages/celery/result.py", line 311, in ready
Nov 05 18:25:17 quarry-web-01 uwsgi-quarry-web[1815]:     return self.state in self.backend.READY_STATES
Nov 05 18:25:17 quarry-web-01 uwsgi-quarry-web[1815]:   File "/srv/quarry/venv/lib/python3.5/site-packages/celery/result.py", line 471, in state
Nov 05 18:25:17 quarry-web-01 uwsgi-quarry-web[1815]:     return self._get_task_meta()['status']
Nov 05 18:25:17 quarry-web-01 uwsgi-quarry-web[1815]:   File "/srv/quarry/venv/lib/python3.5/site-packages/celery/result.py", line 410, in _get_task_meta
Nov 05 18:25:17 quarry-web-01 uwsgi-quarry-web[1815]:     return self._maybe_set_cache(self.backend.get_task_meta(self.id))
Nov 05 18:25:17 quarry-web-01 uwsgi-quarry-web[1815]:   File "/srv/quarry/venv/lib/python3.5/site-packages/celery/backends/base.py", line 365, in get_task_meta
Nov 05 18:25:17 quarry-web-01 uwsgi-quarry-web[1815]:     meta = self._get_task_meta_for(task_id)
Nov 05 18:25:17 quarry-web-01 uwsgi-quarry-web[1815]:   File "/srv/quarry/venv/lib/python3.5/site-packages/celery/backends/base.py", line 680, in _get_task_meta_for
Nov 05 18:25:17 quarry-web-01 uwsgi-quarry-web[1815]:     return self.decode_result(meta)
Nov 05 18:25:17 quarry-web-01 uwsgi-quarry-web[1815]:   File "/srv/quarry/venv/lib/python3.5/site-packages/celery/backends/base.py", line 284, in decode_result
Nov 05 18:25:17 quarry-web-01 uwsgi-quarry-web[1815]:     return self.meta_from_decoded(self.decode(payload))
Nov 05 18:25:17 quarry-web-01 uwsgi-quarry-web[1815]:   File "/srv/quarry/venv/lib/python3.5/site-packages/celery/backends/base.py", line 280, in meta_from_decoded
Nov 05 18:25:17 quarry-web-01 uwsgi-quarry-web[1815]:     meta['result'] = self.exception_to_python(meta['result'])
Nov 05 18:25:17 quarry-web-01 uwsgi-quarry-web[1815]:   File "/srv/quarry/venv/lib/python3.5/site-packages/celery/backends/base.py", line 260, in exception_to_python
Nov 05 18:25:17 quarry-web-01 uwsgi-quarry-web[1815]:     exc = cls(*exc_msg if isinstance(exc_msg, tuple) else exc_msg)
Nov 05 18:25:17 quarry-web-01 uwsgi-quarry-web[1815]: TypeError: __init__() missing 2 required positional arguments: 'params' and 'orig'
Mon, Nov 5, 6:37 PM · Patch-For-Review, Quarry, cloud-services-team (Kanban), Cloud-Services

Sun, Nov 4

zhuyifei1999 added a comment to T153891: generate_family_file.py crashes for a private wiki.

Just find a random private wiki:

Sun, Nov 4, 7:05 PM · Pywikibot

Fri, Nov 2

zhuyifei1999 added a comment to T208561: Allow custom timezone in Toolforge's crontab.

umm, you need that colon https://www.gnu.org/software/libc/manual/html_node/TZ-Variable.html.

Fri, Nov 2, 2:03 AM · Toolforge
zhuyifei1999 added a comment to T208561: Allow custom timezone in Toolforge's crontab.

... which isn't coming to Debian-based distros any time soon

Fri, Nov 2, 1:20 AM · Toolforge
zhuyifei1999 added a comment to T208561: Allow custom timezone in Toolforge's crontab.

That man page seems to be from cronie, the cron implementation used by default on CentOS and RHEL.

Fri, Nov 2, 1:15 AM · Toolforge
zhuyifei1999 added a comment to T208561: Allow custom timezone in Toolforge's crontab.

I'm not sure if it's supported. The cron package we are using is cron/trusty,now 3.0pl1-124ubuntu2 amd64 and its man page don't mention CRON_TZ.

Fri, Nov 2, 1:05 AM · Toolforge
zhuyifei1999 added a comment to T208561: Allow custom timezone in Toolforge's crontab.

https://wikitech.wikimedia.org/wiki/Help:Toolforge/Grid#Specifying_time_zones?

Fri, Nov 2, 12:53 AM · Toolforge

Thu, Nov 1

zhuyifei1999 added a comment to T148792: Send "are you there?" email to tool labs members every 3 months to revalidate email address.

no, not all channels are centrally logged and even if they were how do you properly map back to the relevant LDAP account?

Thu, Nov 1, 11:47 PM · Toolforge, Cloud-Services
zhuyifei1999 added a comment to T148792: Send "are you there?" email to tool labs members every 3 months to revalidate email address.

If the tool is broken (no jobs, or all jobs are in crash loop, infinite sleep, infinite loop, or otherwise provides no service) for a while (say, a year), nobody is willing to take over the maintenance, and nobody from the communities is in any way still using the tool, then yes it's cruft. As you said, they are "tools that simply crash, don't recover and nobody cares". It has nothing to do with whether maintainer is reachable via email.

Thu, Nov 1, 10:53 PM · Toolforge, Cloud-Services
zhuyifei1999 added a comment to T148792: Send "are you there?" email to tool labs members every 3 months to revalidate email address.

It is not uncommon for community members to disappear for say, a few months. Say one tool has only one or two maintainer listed (btw, let's face it: even for tools with lots of maintainers listed usually only a few knows all its internals), and you receive no response after 3 months, what do you do? Stop all the tool jobs to free resources? What if the tool is one quite dependent by the community like flickr2commons or gpsexif? (come on, even these two are are single-maintainer-ed.) And then nobody can invoke abandoned tool policy until the tool is down for 2 weeks, so you get at least 2 weeks of downtime, assuming there is someone to is willing to take over the maintenance, for some arbitrary maintainer-activity criteria we imposed.,

Thu, Nov 1, 7:43 PM · Toolforge, Cloud-Services

Wed, Oct 31

zhuyifei1999 changed the status of T208463: _tkinter in PAWS from Resolved to Declined.
Wed, Oct 31, 10:18 PM · Pywikibot, PAWS
zhuyifei1999 added a comment to T208463: _tkinter in PAWS.

As I understand, Tkinter is for making GUIs, but there is no way currently to forward GUI to the browser client.

Wed, Oct 31, 7:47 PM · Pywikibot, PAWS
zhuyifei1999 added a comment to T207677: Migrate 'Quarry' project to eqiad1.

I added a maintenance message:

MAINTENANCE_MSG: 'Quarrt will be down for maintenance on Monday, November 5 at 5 PM UTC. (T207677)'
Wed, Oct 31, 3:01 PM · Patch-For-Review, Quarry, cloud-services-team (Kanban), Cloud-Services
zhuyifei1999 closed T208411: Check whether quarry project requires NFS or not as Invalid.

Yes T178520: Find somewhere else (not NFS) to store Quarry's resultsets

Wed, Oct 31, 1:46 PM · Quarry, Cloud-VPS
zhuyifei1999 closed T208411: Check whether quarry project requires NFS or not, a subtask of T102240: Audit projects' use of NFS, and remove it where not necessary, as Invalid.
Wed, Oct 31, 1:46 PM · Wikimedia-Incident, Labs-Sprint-106, Labs-Sprint-105, Labs-Sprint-104, Incident-20150617-LabsNFSOutage, Labs-Sprint-103, Labs-Sprint-102, Cloud-Services

Tue, Oct 30

zhuyifei1999 added a comment to T208357: toolforge - Deprecate BigBrother in Grid Engine.

So what is the alternative, in the case of (son of) grid engine? I understand k8s has such feature built-in, but for many wiki-editing bots that has many tasks, running on grid is significantly easier than running on k8s, and many such bots (eg. those that listens on eventstreams) need a way to ensure they are continuously running.

Tue, Oct 30, 11:07 PM · Toolforge, cloud-services-team (Kanban)

Mon, Oct 29

zhuyifei1999 added a parent task for T204422: Install OCRmyPDF dependencies on Tools: T55704: Packages to be added to toollabs puppet.
Mon, Oct 29, 4:38 PM · Toolforge, Wikisource
zhuyifei1999 added a subtask for T55704: Packages to be added to toollabs puppet: T204422: Install OCRmyPDF dependencies on Tools.
Mon, Oct 29, 4:37 PM · Cloud-Services, Tracking, Toolforge

Fri, Oct 26

zhuyifei1999 added a comment to T207677: Migrate 'Quarry' project to eqiad1.

11AM-1PM sounds good to me. @Framawiki is it okay for you? (if not, are you fine with me handling it?)

Fri, Oct 26, 10:06 PM · Patch-For-Review, Quarry, cloud-services-team (Kanban), Cloud-Services
zhuyifei1999 added a comment to T207677: Migrate 'Quarry' project to eqiad1.

That should work, though I will probably be off 10-11 AM, 1-2PM, & 3-5PM Central Time

Fri, Oct 26, 10:00 PM · Patch-For-Review, Quarry, cloud-services-team (Kanban), Cloud-Services

Thu, Oct 25

zhuyifei1999 changed the status of T207980: Non-functional namespaces when entering two points on Commons from Resolved to Invalid.
Thu, Oct 25, 7:39 PM · Commons

Wed, Oct 24

zhuyifei1999 added a comment to T207877: tool enwp10 is not working.

For future reference, T104799 might be related. I see 286 CLOSE_WAIT in lsof -p 31837

Wed, Oct 24, 7:29 PM · Tools
zhuyifei1999 added a comment to T207877: tool enwp10 is not working.
root@tools-webgrid-lighttpd-1411:~# ps uf -u tools.enwp10
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
tools.e+ 31837  0.1  0.5 149200 46948 ?        Ss   Oct22   3:41 /usr/sbin/lighttpd -f /var/run/lighttpd/enwp10 -D
tools.e+ 31844  0.0  0.2 340560 20732 ?        Ss   Oct22   0:00  \_ /usr/bin/php-cgi
tools.e+ 31845  0.0  0.0 340560  6464 ?        S    Oct22   0:00  |   \_ /usr/bin/php-cgi
tools.e+ 31846  0.0  0.0 340560  6464 ?        S    Oct22   0:00  |   \_ /usr/bin/php-cgi
tools.e+ 31847  0.0  0.2 340560 20736 ?        Ss   Oct22   0:00  \_ /usr/bin/php-cgi
tools.e+ 31848  0.0  0.0 340560  6468 ?        S    Oct22   0:00  |   \_ /usr/bin/php-cgi
tools.e+ 31849  0.0  0.0 340560  6468 ?        S    Oct22   0:00  |   \_ /usr/bin/php-cgi
tools.e+ 31851  0.0  0.2 138000 17196 ?        S    Oct22   0:18  \_ /usr/bin/perl /data/project/enwp10/src/public_html/bin/log.fcgi
tools.e+  8729  0.0  0.2 138480 18020 ?        S    09:51   0:22  \_ /usr/bin/perl /data/project/enwp10/src/public_html/bin/list2.fcgi
root@tools-webgrid-lighttpd-1411:~# lsof -p 31837 -a -i | grep LISTEN
lighttpd 31837 tools.enwp10    4u  IPv4 105744188      0t0  TCP *:53758 (LISTEN)
root@tools-webgrid-lighttpd-1411:~# curl http://localhost:53758/enwp10/

In another session:

root@tools-webgrid-lighttpd-1411:~# strace -p 31837
Process 31837 attached
epoll_wait(6, {}, 1025, 1000)           = 0
epoll_wait(6, {}, 1025, 1000)           = 0
epoll_wait(6, {}, 1025, 1000)           = 0
epoll_wait(6, {}, 1025, 1000)           = 0
epoll_wait(6, {}, 1025, 1000)           = 0
epoll_wait(6, {}, 1025, 1000)           = 0
epoll_wait(6, {}, 1025, 1000)           = 0
epoll_wait(6, {}, 1025, 1000)           = 0
Wed, Oct 24, 7:20 PM · Tools
zhuyifei1999 updated subscribers of T207877: tool enwp10 is not working.

CC maintainers @Audiodude @Kelson

Wed, Oct 24, 7:12 PM · Tools
zhuyifei1999 added a comment to T207769: Request for temporary CR+2 rights in Pywikibot core repo.

What GCI tasks are there? I'm slightly concerned about quality, things went messy last year.

Wed, Oct 24, 2:15 AM · Pywikibot, User-D3r1ck01
zhuyifei1999 added a comment to T206710: admin docs: quarry.

It's part of Data Services because of Yuvi I think :) And Quarry has so many users that it's almost production-like.

Wed, Oct 24, 2:06 AM · Quarry, Documentation

Tue, Oct 23

zhuyifei1999 added a comment to T207677: Migrate 'Quarry' project to eqiad1.

@zhuyifei1999 can you add the sql query that you've used in last deployment window to the doc ?

FLUSH TABLES WITH READ LOCK;
SET GLOBAL read_only = 1;
SET GLOBAL read_only = 0;
UNLOCK TABLES;
Tue, Oct 23, 5:08 PM · Patch-For-Review, Quarry, cloud-services-team (Kanban), Cloud-Services

Mon, Oct 22

zhuyifei1999 updated subscribers of T207677: Migrate 'Quarry' project to eqiad1.

The main thing would be copying over the db and creating the config file (containing the secrets). Everything else should be automatic.

Mon, Oct 22, 10:59 PM · Patch-For-Review, Quarry, cloud-services-team (Kanban), Cloud-Services
zhuyifei1999 added a comment to T114401: allow tool users to attach strace to their processes (at least on exec hosts).

I did a cursory search for recent exploits that were related ptrace:

Mon, Oct 22, 10:46 PM · cloud-services-team (Kanban), Toolforge
zhuyifei1999 added a comment to T114401: allow tool users to attach strace to their processes (at least on exec hosts).

If you search for "ptrace escape container", you may start to share my concerns. The Linux security mechanisms are porous and I wouldn't trust simple same-process-UID restrictions considering there have been other methods to abuse ptrace and increase privileges.

Mon, Oct 22, 7:49 PM · cloud-services-team (Kanban), Toolforge
zhuyifei1999 added a comment to T114401: allow tool users to attach strace to their processes (at least on exec hosts).

In the context of the tools themselves, I don't see a problem with secret data. However, the tools nodes do have other processes that might be carrying such secret information in memory for cluster operations.

Mon, Oct 22, 4:23 PM · cloud-services-team (Kanban), Toolforge

Sun, Oct 21

zhuyifei1999 closed T126888: Include query execution time as Resolved.
Sun, Oct 21, 4:19 PM · Patch-For-Review, Quarry
zhuyifei1999 closed T126888: Include query execution time, a subtask of T71264: Show execution time in the Recent Queries page, as Resolved.
Sun, Oct 21, 4:19 PM · Patch-For-Review, Google-Code-in-2018, Quarry

Oct 19 2018

zhuyifei1999 added a comment to T204499: cloudvps: commonsarchive project trusty deprecation.

Ok I think I got it mostly working... until the upload stage...

Oct 19 2018, 2:00 AM · Cloud-VPS (Ubuntu Trusty Deprecation)

Oct 18 2018

zhuyifei1999 merged T207351: OAuthAuthentication throws 'Call to undefined method LoginForm::successfulLogin()' after login into T110460: Update OAuthAuthentication to use AuthManager.
Oct 18 2018, 6:29 PM · User-Tgr, Patch-For-Review, MediaWiki-extensions-OAuthAuthentication, MediaWiki-Authentication-and-authorization
zhuyifei1999 merged task T207351: OAuthAuthentication throws 'Call to undefined method LoginForm::successfulLogin()' after login into T110460: Update OAuthAuthentication to use AuthManager.
Oct 18 2018, 6:29 PM · MediaWiki-extensions-OAuthAuthentication
zhuyifei1999 added a comment to T207351: OAuthAuthentication throws 'Call to undefined method LoginForm::successfulLogin()' after login.

I added a very hacky patch to make this error message disappear:

Oct 18 2018, 6:28 PM · MediaWiki-extensions-OAuthAuthentication
zhuyifei1999 updated subscribers of T204499: cloudvps: commonsarchive project trusty deprecation.

@Nemo_bis I'm thinking of using WikiTeam to backup this wiki to archive.org. Do you think that would work? And could you assist with that?

Oct 18 2018, 5:34 PM · Cloud-VPS (Ubuntu Trusty Deprecation)
zhuyifei1999 edited projects for T175300: OAuth token broken for OAuth file uploader, added: Tools; removed MediaWiki-extensions-OAuthAuthentication.
Oct 18 2018, 5:26 AM · Tools
zhuyifei1999 added a comment to T204499: cloudvps: commonsarchive project trusty deprecation.

I upgraded the main server to stretch. If there are no major issues (there may be some minor issues like T207351), and after I figure out a better way to take backups without needing a separate server, I'll delete the old trusty instances.

Oct 18 2018, 5:00 AM · Cloud-VPS (Ubuntu Trusty Deprecation)
zhuyifei1999 updated subscribers of T207351: OAuthAuthentication throws 'Call to undefined method LoginForm::successfulLogin()' after login.

Searching for this method I see rMW73e23d6643fe99ef71ccf29b6ff7dcd713099cbb (linked to T110277). @Tgr Could you look into this?

Oct 18 2018, 4:35 AM · MediaWiki-extensions-OAuthAuthentication
zhuyifei1999 created T207351: OAuthAuthentication throws 'Call to undefined method LoginForm::successfulLogin()' after login.
Oct 18 2018, 4:32 AM · MediaWiki-extensions-OAuthAuthentication

Oct 16 2018

zhuyifei1999 added a comment to T207098: Example queries for Quarry.

https://wikitech.wikimedia.org/wiki/Help:MySQL_queries#Example_queries ?

Oct 16 2018, 5:01 AM · Documentation, Quarry
zhuyifei1999 added a comment to T126888: Include query execution time.

I guess we could just change to execute + fetch + store. Beware though, that can be heavily lagged due to the use of SQLite on NFS for storing the results.

Oct 16 2018, 4:53 AM · Patch-For-Review, Quarry
zhuyifei1999 added a comment to T126888: Include query execution time.

Tested with https://quarry.wmflabs.org/query/30399:

Oct 16 2018, 4:37 AM · Patch-For-Review, Quarry
zhuyifei1999 reopened T126888: Include query execution time as "Open".

@Framawiki: I think using using the time of cur.execute to get the execution time is flawed. There is no way this query could finish in 0.01 seconds.

Oct 16 2018, 4:27 AM · Patch-For-Review, Quarry
zhuyifei1999 reopened T126888: Include query execution time, a subtask of T71264: Show execution time in the Recent Queries page, as Open.
Oct 16 2018, 4:26 AM · Patch-For-Review, Google-Code-in-2018, Quarry

Oct 15 2018

zhuyifei1999 awarded T126888: Include query execution time a Like token.
Oct 15 2018, 6:58 PM · Patch-For-Review, Quarry
zhuyifei1999 added a comment to T188538: Implement SQL Query Validator in Quarry.

The problem is, this isn't really standardized SQL, but MySQL / MariaDB variant. Upstreaming this sounds unlikely.

Oct 15 2018, 4:19 PM · Quarry

Oct 10 2018

zhuyifei1999 added a comment to T189879: Set up a Nominatim instance to avoid geocode lookups with Google and/or OSM from tools.

Double compression I believe.

Oct 10 2018, 7:14 PM · Tools, Maps (Maps-data)
zhuyifei1999 added a comment to T206616: Please update wget on Toolforge.

If this isn't going to be done due to the version of Ubuntu, is there another way this could be resolved?

Oct 10 2018, 1:39 PM · Toolforge, Internet-Archive

Oct 9 2018

zhuyifei1999 added a comment to T126888: Include query execution time.

It was probably T172143: Weird race condition makes query stuck in queued forever. It would stay pending anyhow.

Oct 9 2018, 2:41 AM · Patch-For-Review, Quarry

Oct 8 2018

zhuyifei1999 added a comment to T206482: Show query code revisions and runs history.

See also T137650: Add links to past query runs

Oct 8 2018, 7:12 PM · Quarry
zhuyifei1999 merged T141233: Mutiple columns with the same name will cause the result to not be shown into T170464: Quarry cannot store results with identical column names.
Oct 8 2018, 7:09 PM · Quarry
zhuyifei1999 merged task T141233: Mutiple columns with the same name will cause the result to not be shown into T170464: Quarry cannot store results with identical column names.
Oct 8 2018, 7:09 PM · Quarry
zhuyifei1999 moved T196153: Quarry cannot save queries with emojies from Backlog to Bugs on the Quarry board.
Oct 8 2018, 7:04 PM · I18n, Quarry
zhuyifei1999 merged T206480: Store and show query execution time into T126888: Include query execution time.
Oct 8 2018, 7:02 PM · Patch-For-Review, Quarry
zhuyifei1999 merged task T206480: Store and show query execution time into T126888: Include query execution time.
Oct 8 2018, 7:02 PM · Quarry
zhuyifei1999 added a comment to T206432: delete.py does not work on non WMF wikis.

T71283: dualism between user and sysop needs to be overtaken FWIW

Oct 8 2018, 4:49 AM · Pywikibot-Other-scripts, Pywikibot
zhuyifei1999 renamed T206432: delete.py does not work on non WMF wikis from delete.py no work on non WMF wikis to delete.py does not work on non WMF wikis.
Oct 8 2018, 4:47 AM · Pywikibot-Other-scripts, Pywikibot

Oct 7 2018

zhuyifei1999 created T206429: tools-k8s-master-01 etcd fail loop.
Oct 7 2018, 10:45 PM · cloud-services-team, Toolforge
zhuyifei1999 added a comment to T194859: Toolforge maintain-kubeusers doesn't fail well when LDAP servers are unreachable.

It lost ldap connection days ago and eventually stopped restarting itself.

Oct 7 2018, 10:14 PM · cloud-services-team (Kanban), Toolforge
zhuyifei1999 renamed T194859: Toolforge maintain-kubeusers doesn't fail well when LDAP servers are unreachable from Toolforge maintain-kubeusers stauck in infinite sleeps of 10 seconds to Toolforge maintain-kubeusers stuck in infinite sleeps of 10 seconds.
Oct 7 2018, 9:59 PM · cloud-services-team (Kanban), Toolforge
zhuyifei1999 reopened T194859: Toolforge maintain-kubeusers doesn't fail well when LDAP servers are unreachable as "Open".

This happened again.

Oct 7 2018, 9:58 PM · cloud-services-team (Kanban), Toolforge

Oct 5 2018

zhuyifei1999 raised the priority of T174498: Monumental tool failed to load bundle.min.js from High to Needs Triage.

Not related.

Oct 5 2018, 1:33 PM · Monumental

Oct 2 2018

zhuyifei1999 reopened T154546: RFC: Support dictionary localization for datasets as "Open".

Reopening per https://lists.wikimedia.org/pipermail/wikitech-l/2018-October/090890.html

Oct 2 2018, 5:01 PM · Maps (Kartographer), Commons-Datasets

Oct 1 2018

RandomDSdevel awarded T203583: {{subst:REVISIONUSER}} no longer substitutes into the current user name, but the username of the last revision a Grey Medal token.
Oct 1 2018, 11:47 PM · MW-1.32-notes (WMF-deploy-2018-09-04 (1.32.0-wmf.20)), User-notice, Patch-For-Review, Multi-Content-Revisions (MCR-SDC File Caption Support - phase 2), Regression, MediaWiki-Parser, MediaWiki-Page-editing
zhuyifei1999 added a comment to T205813: Document mail relay service which can be used to send outbound email from Toolforge Kubernetes containers.

Good question. I believe in the case of grid nodes and bastions it is localhost:

root@tools-exec-1401:~# ps ufww -u Debian-exim
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
Debian-+  1320  0.0  0.0  50216   676 ?        Ss   May11   0:39 /usr/sbin/exim4 -bd -q10m
root@tools-exec-1401:~# lsof -i -a -p 1320
COMMAND  PID        USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
exim4   1320 Debian-exim    3u  IPv4   8086      0t0  TCP localhost:smtp (LISTEN)
exim4   1320 Debian-exim    4u  IPv6   8087      0t0  TCP ip6-localhost:smtp (LISTEN)

It does get tricky in the case of k8s since these listen on localhost, and containers can't access the host.

Oct 1 2018, 7:06 AM · Documentation, Mail, Toolforge
zhuyifei1999 added a comment to T205813: Document mail relay service which can be used to send outbound email from Toolforge Kubernetes containers.

Other images don't have these commands either:

tools.zhuyifei1999-test@tools-bastion-02:~$ webservice --backend kubernetes python shell
If you don't see a command prompt, try pressing enter.
tools.zhuyifei1999-test@interactive:~$ 
tools.zhuyifei1999-test@interactive:~$ which exim
tools.zhuyifei1999-test@interactive:~$ which mail
tools.zhuyifei1999-test@interactive:~$
Oct 1 2018, 5:47 AM · Documentation, Mail, Toolforge