images/.htaccess breaks TransformVia404 functionality
Closed, ResolvedPublic

Description

Steps to reproduce

  1. start with a standard MediaWiki setup
  2. set $wgGenerateThumbnailOnParse = false;
  3. create a .htaccess file in the MediaWiki root directory
  4. copy these rules into it:
RewriteCond %{DOCUMENT_ROOT}%{REQUEST_FILENAME} !-f
RewriteCond %{DOCUMENT_ROOT}%{REQUEST_FILENAME} !-d
RewriteCond %{DOCUMENT_ROOT}%{REQUEST_FILENAME} !-l
RewriteRule ^images/thumb/[0-9a-f]/[0-9a-f][0-9a-f]/[^/]+/[^/]+$ /thumb_handler.php [L,QSA]

RewriteCond %{DOCUMENT_ROOT}%{REQUEST_FILENAME} !-f
RewriteCond %{DOCUMENT_ROOT}%{REQUEST_FILENAME} !-d
RewriteCond %{DOCUMENT_ROOT}%{REQUEST_FILENAME} !-l
RewriteRule ^images/thumb/archive/[0-9a-f]/[0-9a-f][0-9a-f]/[^/]+/[^/]+$ /thumb_handler.php [L,QSA]

...and images will break because these rules are disabled by the lower-level .htaccess file. Is this correct?

Original description

I have enabled TransformVia404 by $wgGenerateThumbnailOnParse = false; and it's works, BUT without images/.htaccess. With images/.htaccess new images return 404 Not Found, but previously generated images (via TransformVia404) work!

Description: Ubuntu 12.04.4 LTS
Server version: Apache/2.2.22 (Ubuntu)
Server built: Mar 19 2014 21:11:10

Contents of file:

# Protect against bug 28235
<IfModule rewrite_module>
        RewriteEngine On
        RewriteCond %{QUERY_STRING} \.[^\\/:*?\x22<>|%]+(#|\?|$) [nocase]
        RewriteRule . - [forbidden]
</IfModule>

I'm not so good in mod_rewrite so don't know what actually wrong.

I'm also tried to delete from /.htaccess all rules except required for TransformVia404 with no success:

RewriteEngine On

RewriteCond %{DOCUMENT_ROOT}%{REQUEST_FILENAME} !-f
RewriteCond %{DOCUMENT_ROOT}%{REQUEST_FILENAME} !-d
RewriteCond %{DOCUMENT_ROOT}%{REQUEST_FILENAME} !-l
RewriteRule ^images/thumb/[0-9a-f]/[0-9a-f][0-9a-f]/[^/]+/[^/]+$ /thumb_handler.php [L,QSA]

RewriteCond %{DOCUMENT_ROOT}%{REQUEST_FILENAME} !-f
RewriteCond %{DOCUMENT_ROOT}%{REQUEST_FILENAME} !-d
RewriteCond %{DOCUMENT_ROOT}%{REQUEST_FILENAME} !-l
RewriteRule ^images/thumb/archive/[0-9a-f]/[0-9a-f][0-9a-f]/[^/]+/[^/]+$ /thumb_handler.php [L,QSA]

Version: 1.23.0
Severity: normal

Details

Reference
bz65220
bzimport raised the priority of this task from to Normal.
bzimport set Reference to bz65220.
bzimport added a subscriber: Unknown Object (MLST).
Scnd created this task.May 12 2014, 11:57 AM
Scnd added a comment.May 12 2014, 12:22 PM

Changed to:

Protect against bug 28235

<IfModule rewrite_module>
RewriteEngine On
RewriteCond %{DOCUMENT_ROOT}%{QUERY_STRING} \.[^\\/:*?\x22<>|%]+(#|\?|$) [nocase]
RewriteRule . - [forbidden]
</IfModule>

Now new images return 403 Forbidden.

Scnd added a comment.May 12 2014, 12:44 PM

(In reply to forwardin from comment #1)

Changed to:

  1. Protect against bug 28235 <IfModule rewrite_module> RewriteEngine On RewriteCond %{DOCUMENT_ROOT}%{QUERY_STRING} \.[^\\/:*?\x22<>|%]+(#|\?|$) [nocase] RewriteRule . - [forbidden] </IfModule>

    Now new images return 403 Forbidden.

Correction: looks like 403 Forbidden on all images. So without %{DOCUMENT_ROOT} this .htaccess working strange for me. But with he works and redirects all images to 403 Forbidden. :(

Tgr added a comment.May 12 2014, 5:38 PM

mediawiki-vagrant with the multimedia role enabled uses 404 transforms, you compare it to your own setup and find out what's being done differently.

Scnd added a comment.May 12 2014, 5:59 PM

With this work:

RewriteEngine On

RewriteCond %{QUERY_STRING} \.[^\\/:*?\x22<>|%]+(#|\?|$) [nocase]
RewriteRule . - [forbidden]

RewriteCond %{DOCUMENT_ROOT}%{REQUEST_FILENAME} !-f
RewriteCond %{DOCUMENT_ROOT}%{REQUEST_FILENAME} !-d
RewriteCond %{DOCUMENT_ROOT}%{REQUEST_FILENAME} !-l
RewriteRule ^thumb/[0-9a-f]/[0-9a-f][0-9a-f]/[^/]+/[^/]+$ /thumb_handler.php [L,QSA]

RewriteCond %{DOCUMENT_ROOT}%{REQUEST_FILENAME} !-f
RewriteCond %{DOCUMENT_ROOT}%{REQUEST_FILENAME} !-d
RewriteCond %{DOCUMENT_ROOT}%{REQUEST_FILENAME} !-l
RewriteRule ^thumb/archive/[0-9a-f]/[0-9a-f][0-9a-f]/[^/]+/[^/]+$ /thumb_handler.php [L,QSA]

/images/thumb/7/7d/111.jpg?.fdfdf - 403 Forbidden
/images/thumb/7/7d/111.jpg - 200 Ok

Looks like if in subfolder exists .htaccess then .htaccess in parent folder ignored.

From http://www.helicontech.com/forum/13329-htaccess_rootchild_not_working.html:

Now I understand the problem. This is by design behavior we implemented exactly as it is in Apache. Lower .htaccess files completely owerride upper configurations unless you put "RewriteOptions inherit" into it. If no .htaccess specified, the closest upper one will be executed. Also Apache always executes httpd.conf file. This behavor can be changed of caurse, but we will loose Apache compatibility then.
Also if inherit options is used it means that all directives will be moved into this file from the upper file and will be applied in the context of the lower file. This is also Apache behavior.
So if you write .htaccess and want include rules from upper file you need to use inherit option. If no .htaccess file exists, your upper rules will be applied.

Scnd added a comment.May 12 2014, 6:15 PM

I found solution:

Protect against bug 28235

<IfModule rewrite_module>
RewriteEngine On
RewriteOptions inherit
RewriteCond %{QUERY_STRING} \.[^\\/:*?\x22<>|%]+(#|\?|$) [nocase]
RewriteRule . - [forbidden]
</IfModule>

RewriteOptions inherit - with this option parent /.htaccess will be executed after /images/.htaccess, BUT you need to change in parent path (delete "images/"):

RewriteCond %{DOCUMENT_ROOT}%{REQUEST_FILENAME} !-f
RewriteCond %{DOCUMENT_ROOT}%{REQUEST_FILENAME} !-d
RewriteCond %{DOCUMENT_ROOT}%{REQUEST_FILENAME} !-l
RewriteRule ^thumb/[0-9a-f]/[0-9a-f][0-9a-f]/[^/]+/[^/]+$ /thumb_handler.php [L,QSA]

RewriteCond %{DOCUMENT_ROOT}%{REQUEST_FILENAME} !-f
RewriteCond %{DOCUMENT_ROOT}%{REQUEST_FILENAME} !-d
RewriteCond %{DOCUMENT_ROOT}%{REQUEST_FILENAME} !-l
RewriteRule ^thumb/archive/[0-9a-f]/[0-9a-f][0-9a-f]/[^/]+/[^/]+$ /thumb_handler.php [L,QSA]

Change 132971 had a related patch set uploaded by Scnd:
Bug 65220

https://gerrit.wikimedia.org/r/132971

Change 132971 abandoned by Scnd:
Bug 65220

Reason:
Bad formatting.

https://gerrit.wikimedia.org/r/132971

Change 132974 had a related patch set uploaded by Scnd:
Bug 65220

https://gerrit.wikimedia.org/r/132974

Tgr added a comment.May 12 2014, 9:17 PM

I am not sure I understand what's the actual bug here. Does the lack of rewrite rule inheritance break 404 handlers on a vagrant box with multimedia role enabled? On a clean MediaWiki install? On a specific site?

Could you provide a step-by-step description with expected and actual outcomes?

(Side note: you can update gerrit patches by using git commit --amend, you don't need to create a new patchset to fix errors in the old one. See https://www.mediawiki.org/wiki/Gerrit/Tutorial#Amending_a_change )

Scnd added a comment.May 13 2014, 2:49 PM

(In reply to Tisza Gergő from comment #9)

I am not sure I understand what's the actual bug here. Does the lack of
rewrite rule inheritance break 404 handlers on a vagrant box with multimedia
role enabled? On a clean MediaWiki install? On a specific site?

Could you provide a step-by-step description with expected and actual
outcomes?

(Side note: you can update gerrit patches by using git commit --amend, you
don't need to create a new patchset to fix errors in the old one. See
https://www.mediawiki.org/wiki/Gerrit/Tutorial#Amending_a_change )

This vagrant box uses server files /etc/apache2/site.d/devwiki/000-devwiki:

RewriteEngine On
RewriteRule ^/$ /w/index.php [R=301]

Alias /wiki "/vagrant/mediawiki/index.php"
Alias /w "/vagrant/mediawiki"
Alias /images "/srv/images"
Alias /mediawiki-vagrant.png /var/www/mediawiki-vagrant.png
Alias /favicon.ico /var/www/favicon.ico

/etc/apache2/site.d/devwiki/thumb-php-on-404:

RewriteEngine On

call thumb.php for thumb images not on disk

RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^/images/thumb/[0-9a-f]/[0-9a-f][0-9a-f]/([^/]+)/[^/]*([0-9]+)px-.*$ /w/thumb_handler.php [PT,QSA,B]

call thumb.php for thumb archive images not on disk

RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^/images/thumb/archive/[0-9a-f]/[0-9a-f][0-9a-f]/([^/]+)/[^/]*([0-9]+)px-.*$ /w/thumb_handler.php [PT,QSA,B]

That is terrible for test environment.

Tgr added a comment.May 13 2014, 5:38 PM

So, if I understand correctly, the steps to reproduce are:

  1. start with a standard MediaWiki setup
  2. set $wgGenerateThumbnailOnParse = false;
  3. create a .htaccess file in the MediaWiki root directory
  4. copy these rules into it: RewriteCond %{DOCUMENT_ROOT}%{REQUEST_FILENAME} !-f RewriteCond %{DOCUMENT_ROOT}%{REQUEST_FILENAME} !-d RewriteCond %{DOCUMENT_ROOT}%{REQUEST_FILENAME} !-l RewriteRule ^images/thumb/[0-9a-f]/[0-9a-f][0-9a-f]/[^/]+/[^/]+$ /thumb_handler.php [L,QSA]

    RewriteCond %{DOCUMENT_ROOT}%{REQUEST_FILENAME} !-f RewriteCond %{DOCUMENT_ROOT}%{REQUEST_FILENAME} !-d RewriteCond %{DOCUMENT_ROOT}%{REQUEST_FILENAME} !-l RewriteRule ^images/thumb/archive/[0-9a-f]/[0-9a-f][0-9a-f]/[^/]+/[^/]+$ /thumb_handler.php [L,QSA]

...and images will break because these rules are disabled by the lower-level .htaccess file. Is this correct?

Scnd added a comment.May 13 2014, 5:53 PM

Also

RewriteEngine on

And disabled will be all contents of all high-level .htaccess files. Other correct.

(In reply to Gerrit Notification Bot from comment #8)

Change 132974 had a related patch set uploaded by Scnd:
Bug 65220

https://gerrit.wikimedia.org/r/132974

For the records, this is for 1.23 only.

Change 135543 had a related patch set uploaded by Krinkle:
images/.htaccess breaks TransformVia404 functionality

https://gerrit.wikimedia.org/r/135543

Jdforrester-WMF moved this task from Untriaged to Backlog on the Multimedia board.Sep 4 2015, 6:36 PM
Restricted Application added subscribers: Steinsplitter, Matanya, Aklapper. · View Herald TranscriptSep 4 2015, 6:36 PM
Nemo_bis updated the task description. (Show Details)Dec 12 2015, 9:36 AM
Nemo_bis set Security to None.
Restricted Application added a project: Commons. · View Herald TranscriptDec 12 2015, 9:36 AM
zhuyifei1999 moved this task from Incoming to Backlog on the Commons board.Dec 15 2015, 10:24 AM
Ciencia_Al_Poder closed this task as Resolved.Feb 7 2016, 5:12 PM
Ciencia_Al_Poder assigned this task to Scnd.
Ciencia_Al_Poder edited subscribers, added: Ciencia_Al_Poder; removed: wikibugs-l-list.

Patch rMWb51645de85224e917a03ba8d04e8127fc0076738 merged, should start to work in 1.27 unless someone backports it