Page MenuHomePhabricator

When "scap pull" does a (slow) CDB rebuild, it should tell me that that's what it's doing
Closed, ResolvedPublic

Assigned To
None
Authored By
Catrope
Apr 4 2017, 11:15 PM
Referenced Files
None
Tokens
"Yellow Medal" token, awarded by mmodell."Orange Medal" token, awarded by Krinkle."Mountain of Wealth" token, awarded by zeljkofilipin.

Description

catrope@mwdebug1002:~$ scap pull
23:06:46 Copying to mwdebug1002.eqiad.wmnet from deployment.eqiad.wmnet
23:06:46 Started rsync common
cannot delete non-empty directory: php-1.29.0-wmf.8/cache/l10n
cannot delete non-empty directory: php-1.29.0-wmf.8/cache/l10n
cannot delete non-empty directory: php-1.29.0-wmf.8/cache
cannot delete non-empty directory: php-1.29.0-wmf.8/cache
cannot delete non-empty directory: php-1.29.0-wmf.8
23:06:52 Finished rsync common (duration: 00m 06s)
[3 minutes of blinking cursor]
catrope@mwdebug1002:~$

Looking at top in tree mode, it looks like the 3 minutes of silence were caused by scap cdb-rebuild --no-progress. It's one thing that progress reporting is explicitly disabled, but it's a different issue altogether that it doesn't even tell me that it's doing a CDB rebuild, or at what timestamps it started and finished doing this. Instead, it just prints no output for 3 minutes, and then exits.

Revisions and Commits

rMSCA Scap
Restricted Differential Revision

Event Timeline

thcipriani triaged this task as Medium priority.Apr 24 2017, 3:44 PM
thcipriani moved this task from Needs triage to Debt on the Scap board.
thcipriani subscribed.

I ran into this when testing the new deployment server (naos) the other day and it was disconcerting.

Still happening eg today:

1zfilipin@mwdebug1002:~$ scap pull
213:14:27 Copying to mwdebug1002.eqiad.wmnet from tin.eqiad.wmnet
313:14:27 Started rsync common
4cannot delete non-empty directory: php-1.30.0-wmf.3/cache/l10n
5cannot delete non-empty directory: php-1.30.0-wmf.3/cache/l10n
6cannot delete non-empty directory: php-1.30.0-wmf.3/cache
7cannot delete non-empty directory: php-1.30.0-wmf.3/cache
8cannot delete non-empty directory: php-1.30.0-wmf.3
9cannot delete non-empty directory: php-1.30.0-wmf.1/cache/l10n
10cannot delete non-empty directory: php-1.30.0-wmf.2/cache/l10n
1113:14:32 Finished rsync common (duration: 00m 05s)

hashar claimed this task.

Logs are back :-}

hashar removed hashar as the assignee of this task.

Wrong task

Basically this problem goes away with T99740.

This is annoying to experienced deployers and at least surprising for new deployers, if not scary.

#wikimedia-operations during EU SWAT today
gilles
is it normal for "scap pull" on mwdebug1002 to hang after "14:12:28 Finished rsync common (duration: 00m 03s)"?

demon added a revision: Restricted Differential Revision.Apr 6 2018, 7:07 PM