Page MenuHomePhabricator

Some transcoding fails because of "is not a valid storage path" in swift
Closed, ResolvedPublic

Event Timeline

Yann created this task.Jan 13 2017, 2:06 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJan 13 2017, 2:06 PM
* "mwstore://local-swift-eqiad/local-transcoded/b/b5/Президент_России_—_2016-01-14_—_Заседание_наблюдательного_совета_Агентства_стратегических_инициатив.webm/Президент_России_—_2016-01-14_—_Заседание_наблюдательного_совета_Агентства_стратегических_инициатив.webm.160p.ogv" is not a valid storage path.
* "mwstore://local-swift-codfw/local-transcoded/b/b5/Президент_России_—_2016-01-14_—_Заседание_наблюдательного_совета_Агентства_стратегических_инициатив.webm/Президент_России_—_2016-01-14_—_Заседание_наблюдательного_совета_Агентства_стратегических_инициатив.webm.160p.ogv" is not a valid storage path.

The filename does not seem long enough, at 194 bytes utf-8.

zhuyifei1999 renamed this task from Transcoding fails: name too long? to Some transcoding fails because of "is not a valid storage path" in swift.Jan 13 2017, 3:02 PM
brion added a subscriber: brion.Jan 13 2017, 3:37 PM

Looks related to old bug T57658 -- perhaps the calculation for the fix there is off.

Per the comment:

	 * Swift has problems where the url-encoded version of
	 * the path (ie 'filename.ogv/filename.ogv.720p.webm' )
	 * is greater that > 1024 bytes, so shorten in that case.

The calculation used in the actual comparison before shortening the filename is this:

		if ( strlen( urlencode( $name ) ) * 2 + 12 > 1024 ) {
			return 'transcode' . '.' . $suffix;
		} else {
			return $name . '.' . $suffix;
		}

which here gives us false:

php > echo strlen(urlencode('Президент_России_—_2016-01-14_—_Заседание_наблюдательного_совета_Агентства_стратегических_инициатив.webm')) * 2 + 12;
1022

So TMH thinks it *can* save the filename, hitting 1022 chars which is under the 1024-char limit.

Possibility: I suspect the "b/b5/" might count in that length too, which would push it to 1031 bytes.

Change 331985 had a related patch set uploaded (by Brion VIBBER):
Fix length calculation for shortening transcode filenames

https://gerrit.wikimedia.org/r/331985

Change 331985 merged by jenkins-bot:
Fix length calculation for shortening transcode filenames

https://gerrit.wikimedia.org/r/331985

Change 332009 had a related patch set uploaded (by Paladox):
Fix length calculation for shortening transcode filenames

https://gerrit.wikimedia.org/r/332009

Change 332009 abandoned by Chad:
Fix length calculation for shortening transcode filenames

Reason:
Already included in wmf.8, which is live everywhere.

https://gerrit.wikimedia.org/r/332009

TheDJ closed this task as Resolved.Jan 26 2017, 2:36 PM
TheDJ assigned this task to brion.
TheDJ removed a project: Patch-For-Review.
TheDJ added a subscriber: TheDJ.

Confirmed fixed