Page MenuHomePhabricator

Protocol-relative URLs cause complete breakage of Squid purge during upload, action=purge, etc.
Closed, ResolvedPublic

Description

In LocalFile: purgeCache(), purgeThumbnails() and recordUpload2() all call SquidUpdate::purge(), and apparently the URLs in all three cases are potentially protocol-relative. SquidUpdate has always had support for purging relative URLs, it calls SquidUpdate::expand(), but this function has not been updated to support protocol-relative URLs, so the result is invalid. HTCP packets with URLs like the following have been logged:

commons.wikimedia.orgupload.wikimedia.org/wikipedia/commons/thumb/0/03/Laurel_Caverns_cave.jpg/1024px-Laurel_Caverns_cave.jpg

Users are complaining. I'm going to look at doing a quick live patch.


Version: 1.17.x
Severity: blocker

Details

Reference
bz30792

Event Timeline

bzimport raised the priority of this task from to Needs Triage.Nov 21 2014, 11:48 PM
bzimport set Reference to bz30792.

Index: SquidUpdate.php

  • SquidUpdate.php (revision 96381)

+++ SquidUpdate.php (working copy)
@@ -208,11 +208,15 @@

  • @return string */ static function expand( $url ) {

+ ### WMF EMERGENCY PATCH FOR BUG 30792 -- TS
+ return wfExpandUrl( $url, PROTO_HTTP );
+ /*

		global $wgInternalServer, $wgServer;
		$server = $wgInternalServer !== false ? $wgInternalServer : $wgServer;
		if( $url !== '' && $url[0] == '/' ) {
			return $server . $url;
		}
		return $url;

+ */

	}

}

Hm, I guess we're only setting $wgInternalServer in secure.php but not during normal operation. I'll poke in a minute.

(In reply to comment #3)

Hm, I guess we're only setting $wgInternalServer in secure.php but not during
normal operation. I'll poke in a minute.

Fixed by setting $wgInternalServer to the value of $wgCanonicalServer in CommonSettings.php .

(In reply to comment #4)

(In reply to comment #3)

Hm, I guess we're only setting $wgInternalServer in secure.php but not during
normal operation. I'll poke in a minute.

Fixed by setting $wgInternalServer to the value of $wgCanonicalServer in
CommonSettings.php .

Lies! The issue was something else. Fixed properly in r96437 and deployed to the cluster. I watched the Squid purge stream on tcpdump to make sure no purges for protocol-relative URLs were sent.

Gilles triaged this task as Unbreak Now! priority.Dec 4 2014, 10:10 AM
Gilles moved this task from Untriaged to Done on the Multimedia board.
Gilles lowered the priority of this task from Unbreak Now! to Needs Triage.Dec 4 2014, 11:23 AM