Page MenuHomePhabricator

Write and implement tests for Wikimedia's Apache configuration (redirects.conf, etc.)
Closed, DeclinedPublic

Description

Whenever changing our Apaches configuration, we want to run integration tests to make sure nothing is going to broke. The repo is operations/apaches-config.git

Jeff Green has written a test suite that is sitting somewhere on the cluster.

Tim Starling posted on https://gerrit.wikimedia.org/r/#/c/15720/ a way to test our Apache configurations:

I tested redirects.conf with apache itself:
redirect-test-server.sh:

#!/bin/bash
apache2 -d . \
    -f redirect-test.conf \
    -X

redirect-test.conf:

# vim: syn=apache
LoadModule rewrite_module /usr/lib/apache2/modules/mod_rewrite.so
Listen 8080
ErrorLog /dev/stderr
PidFile apache2.pid
NameVirtualHost *
<VirtualHost *>
    ServerName default
    DocumentRoot /nonexistent
</VirtualHost>
Include redirects.conf

redirect-test.php

<?php
while (( $line = readline( '> ' ) ) !== false ) {
    $url = trim( $line );
    if ( !preg_match( '/^http:\/\//', $url ) ) {
        $url = 'http://' . $url;
    }
    $c = curl_init( $url );
    curl_setopt_array( $c, array(
        CURLOPT_HEADER => true,
        CURLOPT_RETURNTRANSFER => true,
        CURLOPT_PROXY => 'localhost:8080',
    ) );
    $result = curl_exec( $c );
    $info = curl_getinfo( $c );
    if ( $info['http_code'] == 301 || $info['http_code'] == 302 ) {
        $m = false;
        preg_match( '/Location: (.*)\\n/', $result, $m );
        echo "-> {$m[1]}\n";
    } else {
        echo $info['http_code'] . "\n";
    }
    readline_add_history( $line );
}

We could use a vagrant machine to load the Apache conf and then run the above script.

See Also: T57857

Details

Reference
bz43266

Related Objects

StatusAssignedTask
DeclinedNone
DeclinedNone
Resolvedhashar
Resolvedhashar
Resolvedhashar
Resolvedhashar
Resolvedhashar
DuplicateNone
Resolved bd808
DuplicateNone
DeclinedNone
Declinedhashar
Resolvedhashar
DeclinedNone
Declinedchasemp
DeclinedNone
Resolvedcoren
Invalidhashar
Resolvedhashar
Declinedhashar
ResolvedCmjohnson
ResolvedCmjohnson
Resolvedhashar
ResolvedKrinkle
ResolvedKrinkle
ResolvedKrinkle
Resolvedhashar
ResolvedKrinkle
Resolvedhashar
Resolvedhashar
Resolvedhashar
Resolvedhashar
Resolvedhashar
ResolvedAndrew
ResolvedKrinkle
Resolvedhashar
ResolvedKrinkle
ResolvedPaladox
ResolvedLegoktm
ResolvedLegoktm
ResolvedLegoktm
ResolvedLegoktm
ResolvedPaladox
ResolvedLegoktm
ResolvedLegoktm
Resolvedhashar
Resolvedhashar
Resolved yuvipanda

Event Timeline

bzimport raised the priority of this task from to Low.Nov 22 2014, 12:56 AM
bzimport set Reference to bz43266.
bzimport added a subscriber: Unknown Object (MLST).
hashar created this task.Dec 19 2012, 3:27 PM

This is mostly waiting for Vagrant in Jenkins. Maybe that can be the first candidate.

Reedy added a comment.Oct 22 2013, 8:47 PM

https://gerrit.wikimedia.org/r/#/c/15720 doesn't work...

FYI, I added a list of Main Page urls to all special wikis, and all english project homepages too at [operations/mediawiki-config.git] / tests / urls.txt

I assume this bug covers redirects.conf (https://noc.wikimedia.org/conf/redirects.conf)?

Let's do this!

scfc added a comment.Jan 20 2014, 9:32 PM

For the Toolserver redirects to come, I wrote Gerrit change #108467 if that's useful for this bug.

scfc added a comment.Jan 20 2014, 11:44 PM

BTW, this should be possible without Vagrant: Define a convention to determine two port numbers per Jenkins job, patch the Apache configuration like it is already in operations-apache-config.yaml to set up a http server on $PORT1 and a https server on $PORT2, start Apache, run the tests, stop Apache, release the locks/whatever for the ports. The tests should be in the operations/apache-config itself so they can be updated in parallel to the configuration.

scfc added a comment.Jan 21 2014, 5:08 PM

Just found https://wiki.jenkins-ci.org/display/JENKINS/Port+Allocator+Plugin. Antoine, could this be installed in Jenkins please?

Change 108880 had a related patch set uploaded by Tim Landscheidt:
WIP: Add test suite

https://gerrit.wikimedia.org/r/108880

hashar added a comment.May 6 2014, 3:47 PM

Depend son bug 45499 - Jenkins should run tests in disposable sandboxes

Change 108880 abandoned by Tim Landscheidt:
WIP: Add test suite

https://gerrit.wikimedia.org/r/108880

demon removed a subscriber: demon.Dec 16 2014, 6:05 PM
MZMcBride updated the task description. (Show Details)Jul 16 2015, 12:44 AM
MZMcBride set Security to None.

I don't seem to have permission to view https://gerrit.wikimedia.org/r/#/c/15720/. Weird.

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJul 21 2015, 4:31 AM
jayvdb added a subscriber: jayvdb.Jan 27 2016, 1:21 AM
Krinkle removed a subscriber: Krinkle.Feb 23 2016, 4:50 PM
greg added a comment.Sep 29 2016, 7:42 PM

This follow-up task from an incident report has not been updated recently. If it is no longer valid, please add a comment explaining why. If it is still valid, please prioritize it appropriately relative to your other work. If you have any questions, feel free to ask me (Greg Grossmeier).

hashar closed this task as Declined.Sep 30 2016, 7:33 AM

That is a 3+ years idea and lot has changed since then. I definitely do not have time to lead/write/imagine such a tesdt suite, lets consider the idea moot.