Page MenuHomePhabricator

Server-side upload request for Hurtigruten minutt for minutt videos
Closed, ResolvedPublic

Description

@Meisam asked me to do the transcodes, so here they are (total 161 files):

Uploaded:

The file description pages are already created by @Meisam, see https://commons.wikimedia.org/wiki/Category:Hurtigruten_minutt_for_minutt

Btw, I have attempted async chunked uploading, but even with many many retries the file description page just refuse to acknowledge the file is already uploaded. Eg. https://upload.wikimedia.org/wikipedia/commons/0/0b/Hurtigruten.01.01.1920x1080.NRK2.webm Might worth investigating in a separate task.

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMay 12 2019, 11:01 PM
Reedy updated the task description. (Show Details)May 13 2019, 12:51 AM
Reedy updated the task description. (Show Details)May 13 2019, 1:08 AM
Reedy updated the task description. (Show Details)May 14 2019, 3:36 PM
Reedy updated the task description. (Show Details)May 14 2019, 5:01 PM
zhuyifei1999 renamed this task from Server-side upload request for Zhuyifei1999 to Server-side upload request for Hurtigruten minutt for minutt videos.May 15 2019, 3:52 AM
zhuyifei1999 updated the task description. (Show Details)
zhuyifei1999 updated the task description. (Show Details)May 15 2019, 3:56 AM
zhuyifei1999 updated the task description. (Show Details)May 18 2019, 6:51 PM
zhuyifei1999 updated the task description. (Show Details)
Reedy updated the task description. (Show Details)May 19 2019, 10:09 AM
zhuyifei1999 updated the task description. (Show Details)May 24 2019, 1:37 AM

Had a delay in the progress because I didn't notice the during the scratch server migration the mount was set read-only.

Urbanecm updated the task description. (Show Details)Jun 1 2019, 12:05 PM
Urbanecm updated the task description. (Show Details)Jun 1 2019, 12:08 PM
Urbanecm added a subscriber: Urbanecm.

Downloading files to mwmaint1002:/home/urbanecm/T223052, will upload later.

Urbanecm updated the task description. (Show Details)Jun 1 2019, 7:51 PM
Urbanecm triaged this task as Low priority.Jun 1 2019, 8:15 PM
Urbanecm added a project: User-Urbanecm.
Urbanecm moved this task from Backlog to Blocked on others on the User-Urbanecm board.
Urbanecm updated the task description. (Show Details)Jun 4 2019, 1:33 PM

Uploaded the images currently marked as ready, will check the boxes later, only a mobile rn.

Urbanecm updated the task description. (Show Details)Jun 5 2019, 7:23 AM
Urbanecm updated the task description. (Show Details)Jun 5 2019, 7:34 AM
Urbanecm updated the task description. (Show Details)Jun 6 2019, 8:37 AM
Urbanecm updated the task description. (Show Details)Jun 15 2019, 8:41 PM
Urbanecm updated the task description. (Show Details)Jun 19 2019, 12:02 PM
Urbanecm updated the task description. (Show Details)
urbanecm@mwmaint1002:~$ mwscript importImages.php --wiki=commonswiki --user=Meisam upload/
Import Images

Importing Hurtigruten.06.01.1920x1080.NRK2.webm...done.
Importing Hurtigruten.05.15.1920x1080.NRK2.webm...failed. (An unknown error occurred in storage backend "local-swift-eqiad".)
Importing Hurtigruten.05.13.1920x1080.NRK2.webm...failed. (An unknown error occurred in storage backend "local-swift-eqiad".)
Importing Hurtigruten.05.12.1920x1080.NRK2.webm...done.
Importing Hurtigruten.06.02.1920x1080.NRK2.webm...failed. (An unknown error occurred in storage backend "local-swift-eqiad".)
Importing Hurtigruten.05.11.1920x1080.NRK2.webm...failed. (An unknown error occurred in storage backend "local-swift-eqiad".)
Importing Hurtigruten.05.14.1920x1080.NRK2.webm...failed. (An unknown error occurred in storage backend "local-swift-eqiad".)
Importing Hurtigruten.05.16.1920x1080.NRK2.webm...done.

Found: 8
Added: 3
Failed: 5
urbanecm@mwmaint1002:~$

Something's wrong. Trying again.

Even worse failure...

urbanecm@mwmaint1002:~$ mwscript importImages.php --wiki=commonswiki --user=Meisam upload/
Import Images

Hurtigruten.06.01.1920x1080.NRK2.webm exists, skipping
Importing Hurtigruten.05.15.1920x1080.NRK2.webm...LocalFileLockError from line 2243 of /srv/mediawiki/php-1.34.0-wmf.8/includes/filerepo/file/LocalFile.php: Could not acquire lock for "mwstore://local-multiwrite/local-public/c/c5/Hurtigruten.05.15.1920x1080.NRK2.webm".
#0 /srv/mediawiki/php-1.34.0-wmf.8/includes/filerepo/file/LocalFile.php(1842): LocalFile->lock()
#1 /srv/mediawiki/php-1.34.0-wmf.8/includes/filerepo/file/LocalFile.php(1816): LocalFile->publishTo()
#2 /srv/mediawiki/php-1.34.0-wmf.8/maintenance/importImages.php(315): LocalFile->publish()
#3 /srv/mediawiki/php-1.34.0-wmf.8/maintenance/doMaintenance.php(99): ImportImages->execute()
#4 /srv/mediawiki/php-1.34.0-wmf.8/maintenance/importImages.php(508): include()
#5 /srv/mediawiki/multiversion/MWScript.php(101): include()
#6 {main}
urbanecm@mwmaint1002:~$

It doesn't look someone else is performing the upload, see below:

urbanecm@mwmaint1002:~$ ps aux | grep import
urbanecm 127782  0.0  0.0  12780   964 pts/9    S+   16:51   0:00 grep import
urbanecm@mwmaint1002:~$

I have to admit I pressed Ctrl+C while running importImages.php with Hurtigruten.05.15.1920x1080.NRK2.webm, however, in my opinion, that should cause the lock to remove.

Don't know what to do next, calling Operations and SRE-swift-storage to help with those.

Lock error disappeared, unknown error one is still in place.

Urbanecm updated the task description. (Show Details)Jun 25 2019, 8:29 PM
Urbanecm changed the task status from Open to Stalled.Jul 1 2019, 10:34 AM

Stalling.

Urbanecm updated the task description. (Show Details)Jul 1 2019, 9:56 PM

Mentioned in SAL (#wikimedia-operations) [2019-07-15T13:23:13Z] <Urbanecm> Running mwscript importImages.php --wiki=commonswiki --user=Meisam /home/urbanecm/T223052

Urbanecm updated the task description. (Show Details)Jul 15 2019, 1:25 PM
Urbanecm changed the task status from Stalled to Open.Jul 15 2019, 1:34 PM

Uploading is unblocked!

Urbanecm updated the task description. (Show Details)Jul 15 2019, 1:34 PM
Urbanecm updated the task description. (Show Details)Jul 15 2019, 2:34 PM
Urbanecm updated the task description. (Show Details)Jul 15 2019, 4:39 PM
Urbanecm updated the task description. (Show Details)
Urbanecm updated the task description. (Show Details)Jul 15 2019, 4:44 PM
Urbanecm updated the task description. (Show Details)
Urbanecm moved this task from Working on to Backlog on the User-Urbanecm board.
Urbanecm moved this task from Backlog to Working on on the User-Urbanecm board.
Urbanecm updated the task description. (Show Details)Jul 15 2019, 5:29 PM
Urbanecm updated the task description. (Show Details)Jul 15 2019, 5:51 PM
Urbanecm updated the task description. (Show Details)Jul 15 2019, 5:55 PM
Urbanecm updated the task description. (Show Details)Jul 15 2019, 6:16 PM
Urbanecm updated the task description. (Show Details)Jul 15 2019, 8:09 PM
Urbanecm updated the task description. (Show Details)Jul 16 2019, 9:46 AM
Urbanecm updated the task description. (Show Details)Jul 16 2019, 10:16 AM
Urbanecm updated the task description. (Show Details)Jul 16 2019, 10:44 PM
Urbanecm moved this task from Working on to Blocked on others on the User-Urbanecm board.

waiting on more files to transcode

Urbanecm updated the task description. (Show Details)Jul 21 2019, 10:39 PM
Urbanecm updated the task description. (Show Details)Jul 22 2019, 11:46 PM

Mentioned in SAL (#wikimedia-operations) [2019-07-22T23:54:50Z] <Urbanecm> Run mwscript importImages.php --wiki=commonswiki --user=Meisam /home/urbanecm/T223052

Urbanecm updated the task description. (Show Details)Jul 27 2019, 8:41 PM
Urbanecm updated the task description. (Show Details)
Urbanecm updated the task description. (Show Details)Jul 29 2019, 4:28 PM
Urbanecm updated the task description. (Show Details)
Urbanecm updated the task description. (Show Details)Jul 30 2019, 10:49 PM
Urbanecm updated the task description. (Show Details)Jul 30 2019, 11:00 PM
Urbanecm updated the task description. (Show Details)Aug 1 2019, 9:05 AM
zhuyifei1999 updated the task description. (Show Details)Aug 7 2019, 2:52 PM

Transcoding all done. Weeeeeee

Thanks @zhuyifei1999! Going to do the rest!

Import started.

Urbanecm closed this task as Resolved.Aug 7 2019, 5:47 PM
Urbanecm claimed this task.
Urbanecm updated the task description. (Show Details)

Finally done!