NFS storage slow on Toolforge?
Closed, ResolvedPublicBUG REPORT
Actions

Assigned To

Authored By

	YFdyh000
	Apr 27 2024, 12:16 PM

Description

Steps to replicate the issue (include links if applicable):

tools.yfdyh-bot@tools-bastion-13:~$ dd if=/dev/zero of=$HOME/testfile bs=500M count=1

What happens?:

524288000 bytes (524 MB, 500 MiB) copied, 16.9742 s, 30.9 MB/s
or
524288000 bytes (524 MB, 500 MiB) copied, 89.2577 s, 5.9 MB/s

This causes some tasks to slow down and also causes https://wikitech.wikimedia.org/wiki/Help:Toolforge/Running_Pywikibot_scripts_(advanced) pwb_venv.sh failed due to timeout.

ERROR: timed out 600 seconds waiting for job 'setup-venv' to complete:

What should have happened instead?:
Fast storage as usual.

Software version (on Special:Version page; skip for WMF-hosted wikis like Wikipedia):

Other information (browser name/version, screenshots, etc.):

Event Timeline

YFdyh000 created this task.Apr 27 2024, 12:16 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 27 2024, 12:16 PM

Reedy added projects: Performance Issue, cloud-services-team.Apr 27 2024, 1:54 PM

taavi added a project: Toolforge.Apr 29 2024, 7:58 AM

Don-vip awarded a token.May 8 2024, 12:40 PM

Raymond_Ndibe added a project: User-Raymond_Ndibe.Aug 21 2024, 2:26 PM

Raymond_Ndibe subscribed.

JJMC89 moved this task from Backlog to Shared Storage on the Data-Services board.Sep 12 2024, 12:47 AM

I'll resolve as it seems it's not happening anymore, please create a new task linking here if you find happens again:

tools.wm-lol@tools-bastion-13:~$ dd if=/dev/zero of=$HOME/testfile bs=500M count=1
1+0 records in
1+0 records out
524288000 bytes (524 MB, 500 MiB) copied, 7.07184 s, 74.1 MB/s
tools.wm-lol@tools-bastion-13:~$ dd if=/dev/zero of=$HOME/testfile bs=500M count=1
1+0 records in
1+0 records out
524288000 bytes (524 MB, 500 MiB) copied, 4.46841 s, 117 MB/s
tools.wm-lol@tools-bastion-13:~$ dd if=/dev/zero of=$HOME/testfile bs=500M count=1
1+0 records in
1+0 records out
524288000 bytes (524 MB, 500 MiB) copied, 5.1555 s, 102 MB/s

NFS storage slow on Toolforge?Closed, ResolvedPublicBUG REPORTActions

Description

Event Timeline

NFS storage slow on Toolforge?
Closed, ResolvedPublicBUG REPORT
Actions