Page MenuHomePhabricator

Server-side upload request for Hurtigruten minutt for minutt videos
Closed, ResolvedPublic

Description

@Meisam asked me to do the transcodes, so here they are (total 161 files):

Uploaded:

The file description pages are already created by @Meisam, see https://commons.wikimedia.org/wiki/Category:Hurtigruten_minutt_for_minutt

Btw, I have attempted async chunked uploading, but even with many many retries the file description page just refuse to acknowledge the file is already uploaded. Eg. https://upload.wikimedia.org/wikipedia/commons/0/0b/Hurtigruten.01.01.1920x1080.NRK2.webm Might worth investigating in a separate task.

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
zhuyifei1999 renamed this task from Server-side upload request for Zhuyifei1999 to Server-side upload request for Hurtigruten minutt for minutt videos.May 15 2019, 3:52 AM
zhuyifei1999 updated the task description. (Show Details)

Had a delay in the progress because I didn't notice the during the scratch server migration the mount was set read-only.

Urbanecm added a subscriber: Urbanecm.

Downloading files to mwmaint1002:/home/urbanecm/T223052, will upload later.

Urbanecm added a project: User-Urbanecm.
Urbanecm moved this task from Backlog to Blocked on others on the User-Urbanecm board.

Uploaded the images currently marked as ready, will check the boxes later, only a mobile rn.

Urbanecm updated the task description. (Show Details)
urbanecm@mwmaint1002:~$ mwscript importImages.php --wiki=commonswiki --user=Meisam upload/
Import Images

Importing Hurtigruten.06.01.1920x1080.NRK2.webm...done.
Importing Hurtigruten.05.15.1920x1080.NRK2.webm...failed. (An unknown error occurred in storage backend "local-swift-eqiad".)
Importing Hurtigruten.05.13.1920x1080.NRK2.webm...failed. (An unknown error occurred in storage backend "local-swift-eqiad".)
Importing Hurtigruten.05.12.1920x1080.NRK2.webm...done.
Importing Hurtigruten.06.02.1920x1080.NRK2.webm...failed. (An unknown error occurred in storage backend "local-swift-eqiad".)
Importing Hurtigruten.05.11.1920x1080.NRK2.webm...failed. (An unknown error occurred in storage backend "local-swift-eqiad".)
Importing Hurtigruten.05.14.1920x1080.NRK2.webm...failed. (An unknown error occurred in storage backend "local-swift-eqiad".)
Importing Hurtigruten.05.16.1920x1080.NRK2.webm...done.

Found: 8
Added: 3
Failed: 5
urbanecm@mwmaint1002:~$

Something's wrong. Trying again.

Even worse failure...

urbanecm@mwmaint1002:~$ mwscript importImages.php --wiki=commonswiki --user=Meisam upload/
Import Images

Hurtigruten.06.01.1920x1080.NRK2.webm exists, skipping
Importing Hurtigruten.05.15.1920x1080.NRK2.webm...LocalFileLockError from line 2243 of /srv/mediawiki/php-1.34.0-wmf.8/includes/filerepo/file/LocalFile.php: Could not acquire lock for "mwstore://local-multiwrite/local-public/c/c5/Hurtigruten.05.15.1920x1080.NRK2.webm".
#0 /srv/mediawiki/php-1.34.0-wmf.8/includes/filerepo/file/LocalFile.php(1842): LocalFile->lock()
#1 /srv/mediawiki/php-1.34.0-wmf.8/includes/filerepo/file/LocalFile.php(1816): LocalFile->publishTo()
#2 /srv/mediawiki/php-1.34.0-wmf.8/maintenance/importImages.php(315): LocalFile->publish()
#3 /srv/mediawiki/php-1.34.0-wmf.8/maintenance/doMaintenance.php(99): ImportImages->execute()
#4 /srv/mediawiki/php-1.34.0-wmf.8/maintenance/importImages.php(508): include()
#5 /srv/mediawiki/multiversion/MWScript.php(101): include()
#6 {main}
urbanecm@mwmaint1002:~$

It doesn't look someone else is performing the upload, see below:

urbanecm@mwmaint1002:~$ ps aux | grep import
urbanecm 127782  0.0  0.0  12780   964 pts/9    S+   16:51   0:00 grep import
urbanecm@mwmaint1002:~$

I have to admit I pressed Ctrl+C while running importImages.php with Hurtigruten.05.15.1920x1080.NRK2.webm, however, in my opinion, that should cause the lock to remove.

Don't know what to do next, calling SRE and SRE-swift-storage to help with those.

Lock error disappeared, unknown error one is still in place.

Urbanecm changed the task status from Open to Stalled.Jul 1 2019, 10:34 AM

Stalling.

Mentioned in SAL (#wikimedia-operations) [2019-07-15T13:23:13Z] <Urbanecm> Running mwscript importImages.php --wiki=commonswiki --user=Meisam /home/urbanecm/T223052

Urbanecm changed the task status from Stalled to Open.Jul 15 2019, 1:34 PM

Uploading is unblocked!

Urbanecm updated the task description. (Show Details)
Urbanecm updated the task description. (Show Details)
Urbanecm moved this task from Working on to Backlog on the User-Urbanecm board.
Urbanecm moved this task from Backlog to Working on on the User-Urbanecm board.
Urbanecm moved this task from Working on to Blocked on others on the User-Urbanecm board.

waiting on more files to transcode

Mentioned in SAL (#wikimedia-operations) [2019-07-22T23:54:50Z] <Urbanecm> Run mwscript importImages.php --wiki=commonswiki --user=Meisam /home/urbanecm/T223052

Urbanecm updated the task description. (Show Details)
Urbanecm updated the task description. (Show Details)
Urbanecm claimed this task.
Urbanecm updated the task description. (Show Details)

Finally done!