Page MenuHomePhabricator

Migrate legacy monolitic httpd .conf files to a Debian-like configuration and avoid IP-based VirtualHost
Closed, ResolvedPublic

Description

It would be awesome to migrate the two current legacy monolithic httpd /etc/httpd/conf/virtual.conf and virtual-le.conf files to a Debian-like configuration e.g. splitted for each domain / for each project and tracking available and enabled websites.

In short splitting this monolithic legacy configuration:

To this Debian-like configuration:

https://phabricator.wikimedia.org/diffusion/WIIN/browse/master/servers/fabula/conf/httpd/

E.g.

https://phabricator.wikimedia.org/diffusion/WIIN/browse/master/servers/fabula/conf/httpd/sites-available/it-openstreetmapitalia-ssl.conf

Also actually our VirtualHost(s) were defined in the following way:

<VirtualHost domain.tld>
    ServerName domain.tld

That way is not optimal because it's resolved internally like:

<VirtualHost IP_ADDRESS_DOMAIN_TLD:80>
    ServerName domain.tld

This means that we are making Apache HTTPd listening on a single interface (handling a single IP address) instead of listening whatever interface.

This also does not simplify some other stuff like T274064: Setup a cute default VirtualHost for the Apache in server fabula.

This is also super-slow because each domain should be resolved before every start/restart etc.

In short this can be fixed adopting this approach:

<VirtualHost *:80>
    ServerName domain.tld

This is the status at the time of writing:

$ httpd -S
VirtualHost configuration:
[::1]:*                wmi-fabula (/etc/httpd/conf/virtual.conf:7)
127.0.0.1:*            wmi-fabula (/etc/httpd/conf/virtual.conf:7)
54.38.36.173:443       is a NameVirtualHost
         default server libertadigitali.wikimedia.it (/etc/httpd/conf/virtual-le-ssl.conf:2)
         port 443 namevhost libertadigitali.wikimedia.it (/etc/httpd/conf/virtual-le-ssl.conf:2)
         port 443 namevhost wikilovemonuments.it (/etc/httpd/conf/virtual-le-ssl.conf:57)
         port 443 namevhost www.wikilovemonuments.it (/etc/httpd/conf/virtual-le-ssl.conf:67)
         port 443 namevhost wikilovesmonuments.wikimedia.it (/etc/httpd/conf/virtual-le-ssl.conf:77)
         port 443 namevhost test.wikimedia.it (/etc/httpd/conf/virtual-le-ssl.conf:132)
         port 443 namevhost smssolidale.wikimedia.it (/etc/httpd/conf/virtual-le-ssl.conf:221)
         port 443 namevhost mb.wikilovesmonuments.it (/etc/httpd/conf/virtual-le-ssl.conf:231)
         port 443 namevhost monzaebrianza.wikilovesmonuments.it (/etc/httpd/conf/virtual-le-ssl.conf:241)
         port 443 namevhost old.wikilovesmonuments.it (/etc/httpd/conf/virtual-le-ssl.conf:267)
         port 443 namevhost valledaosta.wikilovesmonuments.it (/etc/httpd/conf/virtual-le-ssl.conf:305)
         port 443 namevhost vda.wikilovesmonuments.it (/etc/httpd/conf/virtual-le-ssl.conf:338)
         port 443 namevhost www.wikilovesmonuments.it (/etc/httpd/conf/virtual-le-ssl.conf:362)
         port 443 namevhost sostienilacultura.it (/etc/httpd/conf/virtual-le-ssl.conf:419)
         port 443 namevhost www.sostienilacultura.it (/etc/httpd/conf/virtual-le-ssl.conf:429)
         port 443 namevhost cinquepermille.wikimedia.it (/etc/httpd/conf/virtual-le-ssl.conf:466)
         port 443 namevhost collab.wikimedia.it (/etc/httpd/conf/virtual-le-ssl.conf:519)
         port 443 namevhost dona.wikimedia.it (/etc/httpd/conf/virtual-le-ssl.conf:537)
         port 443 namevhost landing.wikimedia.it (/etc/httpd/conf/virtual-le-ssl.conf:589)
         port 443 namevhost tasks.osmit.it (/etc/httpd/sites-enabled/it-osmit-tasks-ssl.conf:8)
         port 443 namevhost framadate.wikimedia.it (/etc/httpd/sites-enabled/it-wikimedia-framadate-ssl.conf:12)
         port 443 namevhost matomo.wikimedia.it (/etc/httpd/sites-enabled/it-wikimedia-matomo-ssl.conf:12)
         port 443 namevhost wikimedia.it (/etc/httpd/sites-enabled/it-wikimedia-ssl.conf:16)
54.38.36.173:*         is a NameVirtualHost
         default server old.wikilovesmonuments.it (/etc/httpd/conf/virtual.conf:14)
         port * namevhost old.wikilovesmonuments.it (/etc/httpd/conf/virtual.conf:14)
         port * namevhost www.wikilovesmonuments.it (/etc/httpd/conf/virtual.conf:40)
         port * namevhost monzaebrianza.wikilovesmonuments.it (/etc/httpd/conf/virtual.conf:86)
         port * namevhost valledaosta.wikilovesmonuments.itA (/etc/httpd/conf/virtual.conf:115)
         port * namevhost landing.wikimedia.it (/etc/httpd/conf/virtual.conf:208)
         port * namevhost cinquepermille.wikimedia.it (/etc/httpd/conf/virtual.conf:212)
         port * namevhost smssolidale.wikimedia.it (/etc/httpd/conf/virtual.conf:259)
         port * namevhost dona.wikimedia.it (/etc/httpd/conf/virtual.conf:297)
         port * namevhost libertadigitali.wikimedia.it (/etc/httpd/conf/virtual.conf:342)
         port * namevhost wikilovesmonuments.wikimedia.it (/etc/httpd/conf/virtual.conf:386)
         port * namevhost test.wikimedia.it (/etc/httpd/conf/virtual.conf:433)
         port * namevhost wikimediaitalia.it (/etc/httpd/conf/virtual.conf:456)
         port * namevhost wikimediaitalia.it (/etc/httpd/conf/virtual.conf:460)
         port * namevhost wikimediaitalia.org (/etc/httpd/conf/virtual.conf:465)
         port * namevhost wikimediaitalia.org (/etc/httpd/conf/virtual.conf:469)
         port * namevhost vda.wikilovesmonuments.it (/etc/httpd/conf/virtual.conf:474)
         port * namevhost mb.wikilovesmonuments.it (/etc/httpd/conf/virtual.conf:485)
         port * namevhost wikilovemonument.it (/etc/httpd/conf/virtual.conf:495)
         port * namevhost www.wikilovemonument.it (/etc/httpd/conf/virtual.conf:499)
         port * namevhost wikilovesmonument.it (/etc/httpd/conf/virtual.conf:503)
         port * namevhost www.wikilovesmonument.it (/etc/httpd/conf/virtual.conf:507)
         port * namevhost wikilovemonuments.it (/etc/httpd/conf/virtual.conf:511)
         port * namevhost www.wikilovemonuments.it (/etc/httpd/conf/virtual.conf:515)
         port * namevhost wikilovesmonuments.it (/etc/httpd/conf/virtual.conf:519)
         port * namevhost www.wikilovesreading.it (/etc/httpd/conf/virtual.conf:523)
         port * namevhost wikilovesreading.it (/etc/httpd/conf/virtual.conf:527)
         port * namevhost www.wikilovesbooks.it (/etc/httpd/conf/virtual.conf:531)
         port * namevhost wikilovesbooks.it (/etc/httpd/conf/virtual.conf:535)
         port * namevhost www.wikilovesreading.org (/etc/httpd/conf/virtual.conf:539)
         port * namevhost wikilovesreading.org (/etc/httpd/conf/virtual.conf:543)
         port * namevhost www.wikilovesbooks.org (/etc/httpd/conf/virtual.conf:547)
         port * namevhost wikilovesbooks.org (/etc/httpd/conf/virtual.conf:551)
         port * namevhost wikilibri.it (/etc/httpd/conf/virtual.conf:555)
         port * namevhost www.wikilibri.it (/etc/httpd/conf/virtual.conf:559)
         port * namevhost wikiscuola.org (/etc/httpd/conf/virtual.conf:563)
         port * namevhost www.wikiscuola.org (/etc/httpd/conf/virtual.conf:567)
         port * namevhost wikiscuola.info (/etc/httpd/conf/virtual.conf:571)
         port * namevhost www.wikiscuola.info (/etc/httpd/conf/virtual.conf:575)
         port * namevhost wlm.wikimedia.it (/etc/httpd/conf/virtual.conf:581)
         port * namevhost www.sostienilacultura.it (/etc/httpd/conf/virtual.conf:621)
         port * namevhost sostienilacultura.it (/etc/httpd/conf/virtual.conf:651)
         port * namevhost collab.wikimedia.it (/etc/httpd/conf/virtual.conf:656)
         port * namevhost wikimedia.info (/etc/httpd/conf/virtual.conf:676)
         port * namevhost www.wikimedia.info (/etc/httpd/conf/virtual.conf:680)
         port * namevhost mediawiki.biz (/etc/httpd/conf/virtual.conf:685)
         port * namevhost www.mediawiki.biz (/etc/httpd/conf/virtual.conf:689)
         port * namevhost mediawiki.it (/etc/httpd/conf/virtual.conf:694)
         port * namevhost www.mediawiki.it (/etc/httpd/conf/virtual.conf:698)
         port * namevhost mediawiki.net (/etc/httpd/conf/virtual.conf:703)
         port * namevhost www.mediawiki.net (/etc/httpd/conf/virtual.conf:707)
         port * namevhost mediawiki.eu (/etc/httpd/conf/virtual.conf:712)
         port * namevhost www.mediawiki.eu (/etc/httpd/conf/virtual.conf:716)
         port * namevhost wikinotizie.it (/etc/httpd/conf/virtual.conf:721)
         port * namevhost www.wikinotizie.it (/etc/httpd/conf/virtual.conf:725)
         port * namevhost itwp.it (/etc/httpd/conf/virtual.conf:730)
         port * namevhost www.itwp.it (/etc/httpd/conf/virtual.conf:734)
         port * namevhost itwp.org (/etc/httpd/conf/virtual.conf:739)
         port * namevhost www.itwp.org (/etc/httpd/conf/virtual.conf:743)
         port * namevhost wikiquote.biz (/etc/httpd/conf/virtual.conf:748)
         port * namevhost www.wikiquote.biz (/etc/httpd/conf/virtual.conf:752)
         port * namevhost wikiquote.eu (/etc/httpd/conf/virtual.conf:757)
         port * namevhost www.wikiquote.eu (/etc/httpd/conf/virtual.conf:761)
         port * namevhost wikibooks.biz (/etc/httpd/conf/virtual.conf:766)
         port * namevhost www.wikibooks.biz (/etc/httpd/conf/virtual.conf:770)
         port * namevhost wikimania.biz (/etc/httpd/conf/virtual.conf:775)
         port * namevhost www.wikimania.biz (/etc/httpd/conf/virtual.conf:779)
         port * namevhost wikimania.info (/etc/httpd/conf/virtual.conf:784)
         port * namevhost www.wikimania.info (/etc/httpd/conf/virtual.conf:788)
         port * namevhost wiktionary.info (/etc/httpd/conf/virtual.conf:793)
         port * namevhost www.wiktionary.info (/etc/httpd/conf/virtual.conf:797)
         port * namevhost wikispecies.it (/etc/httpd/conf/virtual.conf:802)
         port * namevhost www.wikispecies.it (/etc/httpd/conf/virtual.conf:806)
         port * namevhost wikispecies.eu (/etc/httpd/conf/virtual.conf:811)
         port * namevhost www.wikispecies.eu (/etc/httpd/conf/virtual.conf:815)
         port * namevhost commons.it (/etc/httpd/conf/virtual.conf:820)
         port * namevhost www.commons.it (/etc/httpd/conf/virtual.conf:824)
         port * namevhost wikimania.it (/etc/httpd/conf/virtual.conf:829)
         port * namevhost www.wikimania.it (/etc/httpd/conf/virtual.conf:833)
         port * namevhost wikisource.eu (/etc/httpd/conf/virtual.conf:838)
         port * namevhost www.wikisource.eu (/etc/httpd/conf/virtual.conf:842)
         port * namevhost wikicitazioni.it (/etc/httpd/conf/virtual.conf:847)
         port * namevhost www.wikicitazioni.it (/etc/httpd/conf/virtual.conf:851)
         port * namevhost wikiquote.it (/etc/httpd/conf/virtual.conf:856)
         port * namevhost www.wikiquote.it (/etc/httpd/conf/virtual.conf:860)
         port * namevhost wikidati.it (/etc/httpd/conf/virtual.conf:865)
         port * namevhost www.wikidati.it (/etc/httpd/conf/virtual.conf:869)
         port * namevhost wikidata.biz (/etc/httpd/conf/virtual.conf:874)
         port * namevhost www.wikidata.biz (/etc/httpd/conf/virtual.conf:878)
         port * namevhost wikidata.eu (/etc/httpd/conf/virtual.conf:883)
         port * namevhost www.wikidata.eu (/etc/httpd/conf/virtual.conf:887)
         port * namevhost wikivoyage.biz (/etc/httpd/conf/virtual.conf:892)
         port * namevhost www.wikivoyage.biz (/etc/httpd/conf/virtual.conf:896)
         port * namevhost wikivoyage.it (/etc/httpd/conf/virtual.conf:901)
         port * namevhost www.wikivoyage.it (/etc/httpd/conf/virtual.conf:905)
         port * namevhost wikiteca.it (/etc/httpd/conf/virtual.conf:910)
         port * namevhost www.wikiteca.it (/etc/httpd/conf/virtual.conf:914)
         port * namevhost wikisource.biz (/etc/httpd/conf/virtual.conf:919)
         port * namevhost www.wikisource.biz (/etc/httpd/conf/virtual.conf:923)
         port * namevhost xn--wikiversit-q4a.it (/etc/httpd/conf/virtual.conf:928)
         port * namevhost www.xn--wikiversit-q4a.it (/etc/httpd/conf/virtual.conf:932)
         port * namevhost wikiversity.biz (/etc/httpd/conf/virtual.conf:937)
         port * namevhost www.wikiversity.biz (/etc/httpd/conf/virtual.conf:941)
         port * namevhost wikiversity.info (/etc/httpd/conf/virtual.conf:946)
         port * namevhost www.wikiversity.info (/etc/httpd/conf/virtual.conf:950)
         port * namevhost libertadigitali.it (/etc/httpd/conf/virtual.conf:955)
         port * namevhost www.libertadigitali.it (/etc/httpd/conf/virtual.conf:959)
         port * namevhost stateofthemap.it (/etc/httpd/conf/virtual.conf:964)
         port * namevhost www.stateofthemap.it (/etc/httpd/conf/virtual.conf:968)
         port * namevhost tasks.osmit.it (/etc/httpd/sites-enabled/it-osmit-tasks-txt.conf:12)
         port * namevhost framadate.wikimedia.it (/etc/httpd/sites-enabled/it-wikimedia-framadate-txt.conf:12)
         port * namevhost matomo.wikimedia.it (/etc/httpd/sites-enabled/it-wikimedia-matomo-txt.conf:12)
         port * namevhost wikimedia.it (/etc/httpd/sites-enabled/it-wikimedia-txt.conf:16)
80.211.31.124:*        www.wikimedia.it (/etc/httpd/conf/virtual.conf:144)
80.211.31.124:443      www.wikimedia.it (/etc/httpd/conf/virtual-le-ssl.conf:159)
*:443                  54.38.36.173 (/etc/httpd/conf.d/ssl.conf:56)
ServerRoot: "/etc/httpd"
Main DocumentRoot: "/var/www"
Main ErrorLog: "/etc/httpd/logs/error_log"
Mutex mpm-accept: using_defaults
Mutex fcgid-pipe: using_defaults
Mutex authdigest-opaque: using_defaults
Mutex proxy-balancer-shm: using_defaults
Mutex rewrite-map: using_defaults
Mutex authdigest-client: using_defaults
Mutex fcgid-proctbl: using_defaults
Mutex ssl-stapling: using_defaults
Mutex proxy: using_defaults
Mutex ldap-cache: using_defaults
Mutex authn-socache: using_defaults
Mutex ssl-cache: using_defaults
Mutex default: dir="/run/httpd/" mechanism=default 
PidFile: "/etc/httpd/run/httpd.pid"
Define: _RH_HAS_HTTPPROTOCOLOPTIONS
Define: DUMP_VHOSTS
Define: DUMP_RUN_CFG
User: name="nobody" id=99
Group: name="nogroup" id=65534

Event Timeline

valerio.bozzolan triaged this task as Low priority.
valerio.bozzolan created this task.
valerio.bozzolan updated the task description. (Show Details)
valerio.bozzolan updated the task description. (Show Details)

I started the refactor and I noticed that the reload is much faster now.

Here the current configuration dump. Note the *:PORT instead of the IP:PORT:

$ httpd -S
VirtualHost configuration:
*:443                  is a NameVirtualHost
         default server fabula.wikimedia.it (/etc/httpd/conf.d/ssl.conf:56)
         port 443 namevhost fabula.wikimedia.it (/etc/httpd/conf.d/ssl.conf:56)
         port 443 namevhost libertadigitali.wikimedia.it (/etc/httpd/conf/virtual-le-ssl.conf:2)
         port 443 namevhost www.wikilovemonuments.it (/etc/httpd/conf/virtual-le-ssl.conf:73)
         port 443 namevhost wikilovesmonuments.wikimedia.it (/etc/httpd/conf/virtual-le-ssl.conf:83)
         port 443 namevhost test.wikimedia.it (/etc/httpd/conf/virtual-le-ssl.conf:138)
         port 443 namevhost www.wikimedia.it (/etc/httpd/conf/virtual-le-ssl.conf:165)
         port 443 namevhost smssolidale.wikimedia.it (/etc/httpd/conf/virtual-le-ssl.conf:227)
         port 443 namevhost mb.wikilovesmonuments.it (/etc/httpd/conf/virtual-le-ssl.conf:237)
         port 443 namevhost monzaebrianza.wikilovesmonuments.it (/etc/httpd/conf/virtual-le-ssl.conf:247)
         port 443 namevhost old.wikilovesmonuments.it (/etc/httpd/conf/virtual-le-ssl.conf:273)
         port 443 namevhost valledaosta.wikilovesmonuments.it (/etc/httpd/conf/virtual-le-ssl.conf:311)
         port 443 namevhost www.wikilovesmonuments.it (/etc/httpd/conf/virtual-le-ssl.conf:373)
         port 443 namevhost sostienilacultura.it (/etc/httpd/conf/virtual-le-ssl.conf:431)
         port 443 namevhost www.sostienilacultura.it (/etc/httpd/conf/virtual-le-ssl.conf:441)
         port 443 namevhost cinquepermille.wikimedia.it (/etc/httpd/conf/virtual-le-ssl.conf:478)
         port 443 namevhost collab.wikimedia.it (/etc/httpd/conf/virtual-le-ssl.conf:531)
         port 443 namevhost dona.wikimedia.it (/etc/httpd/conf/virtual-le-ssl.conf:549)
         port 443 namevhost landing.wikimedia.it (/etc/httpd/conf/virtual-le-ssl.conf:601)
         port 443 namevhost tasks.osmit.it (/etc/httpd/sites-enabled/it-osmit-tasks-ssl.conf:8)
         port 443 namevhost framadate.wikimedia.it (/etc/httpd/sites-enabled/it-wikimedia-framadate-ssl.conf:12)
         port 443 namevhost matomo.wikimedia.it (/etc/httpd/sites-enabled/it-wikimedia-matomo-ssl.conf:12)
         port 443 namevhost wikimedia.it (/etc/httpd/sites-enabled/it-wikimedia-ssl.conf:15)
*:80                   is a NameVirtualHost
         default server fabula.wikimedia.it (/etc/httpd/conf/virtual.conf:12)
         port 80 namevhost fabula.wikimedia.it (/etc/httpd/conf/virtual.conf:12)
         port 80 namevhost old.wikilovesmonuments.it (/etc/httpd/conf/virtual.conf:20)
         port 80 namevhost www.wikilovesmonuments.it (/etc/httpd/conf/virtual.conf:46)
         port 80 namevhost monzaebrianza.wikilovesmonuments.it (/etc/httpd/conf/virtual.conf:92)
         port 80 namevhost valledaosta.wikilovesmonuments.itA (/etc/httpd/conf/virtual.conf:121)
         port 80 namevhost www.wikimedia.it (/etc/httpd/conf/virtual.conf:150)
         port 80 namevhost landing.wikimedia.it (/etc/httpd/conf/virtual.conf:214)
         port 80 namevhost cinquepermille.wikimedia.it (/etc/httpd/conf/virtual.conf:218)
         port 80 namevhost smssolidale.wikimedia.it (/etc/httpd/conf/virtual.conf:265)
         port 80 namevhost dona.wikimedia.it (/etc/httpd/conf/virtual.conf:303)
         port 80 namevhost libertadigitali.wikimedia.it (/etc/httpd/conf/virtual.conf:348)
         port 80 namevhost wikilovesmonuments.wikimedia.it (/etc/httpd/conf/virtual.conf:392)
         port 80 namevhost test.wikimedia.it (/etc/httpd/conf/virtual.conf:439)
         port 80 namevhost wikimediaitalia.it (/etc/httpd/conf/virtual.conf:462)
         port 80 namevhost wikimediaitalia.it (/etc/httpd/conf/virtual.conf:466)
         port 80 namevhost wikimediaitalia.org (/etc/httpd/conf/virtual.conf:471)
         port 80 namevhost wikimediaitalia.org (/etc/httpd/conf/virtual.conf:475)
         port 80 namevhost vda.wikilovesmonuments.it (/etc/httpd/conf/virtual.conf:480)
         port 80 namevhost mb.wikilovesmonuments.it (/etc/httpd/conf/virtual.conf:491)
         port 80 namevhost wikilovemonument.it (/etc/httpd/conf/virtual.conf:501)
         port 80 namevhost www.wikilovemonument.it (/etc/httpd/conf/virtual.conf:505)
         port 80 namevhost wikilovesmonument.it (/etc/httpd/conf/virtual.conf:509)
         port 80 namevhost www.wikilovesmonument.it (/etc/httpd/conf/virtual.conf:513)
         port 80 namevhost wikilovemonuments.it (/etc/httpd/conf/virtual.conf:517)
         port 80 namevhost www.wikilovemonuments.it (/etc/httpd/conf/virtual.conf:521)
         port 80 namevhost wikilovesmonuments.it (/etc/httpd/conf/virtual.conf:525)
         port 80 namevhost www.wikilovesreading.it (/etc/httpd/conf/virtual.conf:529)
         port 80 namevhost wikilovesreading.it (/etc/httpd/conf/virtual.conf:533)
         port 80 namevhost www.wikilovesbooks.it (/etc/httpd/conf/virtual.conf:537)
         port 80 namevhost wikilovesbooks.it (/etc/httpd/conf/virtual.conf:541)
         port 80 namevhost www.wikilovesreading.org (/etc/httpd/conf/virtual.conf:545)
         port 80 namevhost wikilovesreading.org (/etc/httpd/conf/virtual.conf:549)
         port 80 namevhost www.wikilovesbooks.org (/etc/httpd/conf/virtual.conf:553)
         port 80 namevhost wikilovesbooks.org (/etc/httpd/conf/virtual.conf:557)
         port 80 namevhost wikilibri.it (/etc/httpd/conf/virtual.conf:561)
         port 80 namevhost www.wikilibri.it (/etc/httpd/conf/virtual.conf:565)
         port 80 namevhost wikiscuola.org (/etc/httpd/conf/virtual.conf:569)
         port 80 namevhost www.wikiscuola.org (/etc/httpd/conf/virtual.conf:573)
         port 80 namevhost wikiscuola.info (/etc/httpd/conf/virtual.conf:577)
         port 80 namevhost www.wikiscuola.info (/etc/httpd/conf/virtual.conf:581)
         port 80 namevhost wlm.wikimedia.it (/etc/httpd/conf/virtual.conf:587)
         port 80 namevhost www.sostienilacultura.it (/etc/httpd/conf/virtual.conf:627)
         port 80 namevhost sostienilacultura.it (/etc/httpd/conf/virtual.conf:657)
         port 80 namevhost collab.wikimedia.it (/etc/httpd/conf/virtual.conf:662)
         port 80 namevhost wikimedia.info (/etc/httpd/conf/virtual.conf:682)
         port 80 namevhost www.wikimedia.info (/etc/httpd/conf/virtual.conf:686)
         port 80 namevhost mediawiki.biz (/etc/httpd/conf/virtual.conf:691)
         port 80 namevhost www.mediawiki.biz (/etc/httpd/conf/virtual.conf:695)
         port 80 namevhost mediawiki.it (/etc/httpd/conf/virtual.conf:700)
         port 80 namevhost www.mediawiki.it (/etc/httpd/conf/virtual.conf:704)
         port 80 namevhost mediawiki.net (/etc/httpd/conf/virtual.conf:709)
         port 80 namevhost www.mediawiki.net (/etc/httpd/conf/virtual.conf:713)
         port 80 namevhost mediawiki.eu (/etc/httpd/conf/virtual.conf:718)
         port 80 namevhost www.mediawiki.eu (/etc/httpd/conf/virtual.conf:722)
         port 80 namevhost wikinotizie.it (/etc/httpd/conf/virtual.conf:727)
         port 80 namevhost www.wikinotizie.it (/etc/httpd/conf/virtual.conf:731)
         port 80 namevhost itwp.it (/etc/httpd/conf/virtual.conf:736)
         port 80 namevhost www.itwp.it (/etc/httpd/conf/virtual.conf:740)
         port 80 namevhost itwp.org (/etc/httpd/conf/virtual.conf:745)
         port 80 namevhost www.itwp.org (/etc/httpd/conf/virtual.conf:749)
         port 80 namevhost wikiquote.biz (/etc/httpd/conf/virtual.conf:754)
         port 80 namevhost www.wikiquote.biz (/etc/httpd/conf/virtual.conf:758)
         port 80 namevhost wikiquote.eu (/etc/httpd/conf/virtual.conf:763)
         port 80 namevhost www.wikiquote.eu (/etc/httpd/conf/virtual.conf:767)
         port 80 namevhost wikibooks.biz (/etc/httpd/conf/virtual.conf:772)
         port 80 namevhost www.wikibooks.biz (/etc/httpd/conf/virtual.conf:776)
         port 80 namevhost wikimania.biz (/etc/httpd/conf/virtual.conf:781)
         port 80 namevhost www.wikimania.biz (/etc/httpd/conf/virtual.conf:785)
         port 80 namevhost wikimania.info (/etc/httpd/conf/virtual.conf:790)
         port 80 namevhost www.wikimania.info (/etc/httpd/conf/virtual.conf:794)
         port 80 namevhost wiktionary.info (/etc/httpd/conf/virtual.conf:799)
         port 80 namevhost www.wiktionary.info (/etc/httpd/conf/virtual.conf:803)
         port 80 namevhost wikispecies.it (/etc/httpd/conf/virtual.conf:808)
         port 80 namevhost www.wikispecies.it (/etc/httpd/conf/virtual.conf:812)
         port 80 namevhost wikispecies.eu (/etc/httpd/conf/virtual.conf:817)
         port 80 namevhost www.wikispecies.eu (/etc/httpd/conf/virtual.conf:821)
         port 80 namevhost commons.it (/etc/httpd/conf/virtual.conf:826)
         port 80 namevhost www.commons.it (/etc/httpd/conf/virtual.conf:830)
         port 80 namevhost wikimania.it (/etc/httpd/conf/virtual.conf:835)
         port 80 namevhost www.wikimania.it (/etc/httpd/conf/virtual.conf:839)
         port 80 namevhost wikisource.eu (/etc/httpd/conf/virtual.conf:844)
         port 80 namevhost www.wikisource.eu (/etc/httpd/conf/virtual.conf:848)
         port 80 namevhost wikicitazioni.it (/etc/httpd/conf/virtual.conf:853)
         port 80 namevhost www.wikicitazioni.it (/etc/httpd/conf/virtual.conf:857)
         port 80 namevhost wikiquote.it (/etc/httpd/conf/virtual.conf:862)
         port 80 namevhost www.wikiquote.it (/etc/httpd/conf/virtual.conf:866)
         port 80 namevhost wikidati.it (/etc/httpd/conf/virtual.conf:871)
         port 80 namevhost www.wikidati.it (/etc/httpd/conf/virtual.conf:875)
         port 80 namevhost wikidata.biz (/etc/httpd/conf/virtual.conf:880)
         port 80 namevhost www.wikidata.biz (/etc/httpd/conf/virtual.conf:884)
         port 80 namevhost wikidata.eu (/etc/httpd/conf/virtual.conf:889)
         port 80 namevhost www.wikidata.eu (/etc/httpd/conf/virtual.conf:893)
         port 80 namevhost wikivoyage.biz (/etc/httpd/conf/virtual.conf:898)
         port 80 namevhost www.wikivoyage.biz (/etc/httpd/conf/virtual.conf:902)
         port 80 namevhost wikivoyage.it (/etc/httpd/conf/virtual.conf:907)
         port 80 namevhost www.wikivoyage.it (/etc/httpd/conf/virtual.conf:911)
         port 80 namevhost wikiteca.it (/etc/httpd/conf/virtual.conf:916)
         port 80 namevhost www.wikiteca.it (/etc/httpd/conf/virtual.conf:920)
         port 80 namevhost wikisource.biz (/etc/httpd/conf/virtual.conf:925)
         port 80 namevhost www.wikisource.biz (/etc/httpd/conf/virtual.conf:929)
         port 80 namevhost xn--wikiversit-q4a.it (/etc/httpd/conf/virtual.conf:934)
         port 80 namevhost www.xn--wikiversit-q4a.it (/etc/httpd/conf/virtual.conf:938)
         port 80 namevhost wikiversity.biz (/etc/httpd/conf/virtual.conf:943)
         port 80 namevhost www.wikiversity.biz (/etc/httpd/conf/virtual.conf:947)
         port 80 namevhost wikiversity.info (/etc/httpd/conf/virtual.conf:952)
         port 80 namevhost www.wikiversity.info (/etc/httpd/conf/virtual.conf:956)
         port 80 namevhost libertadigitali.it (/etc/httpd/conf/virtual.conf:961)
         port 80 namevhost www.libertadigitali.it (/etc/httpd/conf/virtual.conf:965)
         port 80 namevhost stateofthemap.it (/etc/httpd/conf/virtual.conf:970)
         port 80 namevhost www.stateofthemap.it (/etc/httpd/conf/virtual.conf:974)
         port 80 namevhost tasks.osmit.it (/etc/httpd/sites-enabled/it-osmit-tasks-txt.conf:12)
         port 80 namevhost framadate.wikimedia.it (/etc/httpd/sites-enabled/it-wikimedia-framadate-txt.conf:12)
         port 80 namevhost matomo.wikimedia.it (/etc/httpd/sites-enabled/it-wikimedia-matomo-txt.conf:12)
         port 80 namevhost wikimedia.it (/etc/httpd/sites-enabled/it-wikimedia-txt.conf:14)
ServerRoot: "/etc/httpd"
Main DocumentRoot: "/var/www"
Main ErrorLog: "/etc/httpd/logs/error_log"
Mutex fcgid-proctbl: using_defaults
Mutex ssl-stapling: using_defaults
Mutex proxy: using_defaults
Mutex ldap-cache: using_defaults
Mutex authn-socache: using_defaults
Mutex ssl-cache: using_defaults
Mutex default: dir="/run/httpd/" mechanism=default 
Mutex mpm-accept: using_defaults
Mutex fcgid-pipe: using_defaults
Mutex authdigest-opaque: using_defaults
Mutex proxy-balancer-shm: using_defaults
Mutex rewrite-map: using_defaults
Mutex authdigest-client: using_defaults
PidFile: "/etc/httpd/run/httpd.pid"
Define: _RH_HAS_HTTPPROTOCOLOPTIONS
Define: DUMP_VHOSTS
Define: DUMP_RUN_CFG
User: name="nobody" id=99
Group: name="nogroup" id=65534
valerio.bozzolan added a subscriber: Ferdi2005.

I don't know why but @Ferdi2005 was online with me to assist this mega-commit.

The Gerrit→Phabricator sync seems very slow today. Anyway a mega-commit will land here soon. Good night Ferdi! asd

asd

NOTE: I had a good sleep. I'm fine now! asd

I have to report that last night I had an usual little fight with mod_ssl. That module is somehow not very puppy with sysadmins because everytime you run apachectl configtest it's a bit like playing with an asshole poker player: you can get a nice OK but then, if you trust it a bit, proceeding with a pair of gloves running a tender apachectl graceful or systemctl reload httpd or a rustic systemctl restart httpd or whatever... surprise! Your whole webserver is down because mod_ssl.

In short, last night there was a downtime caused by the last night refactor. Here a monitoring reported by @Nemo_bis:

The monitor WMI WLM (https://wikilovesmonuments.wikimedia.it/) is back UP (HTTP 200 - OK) (It was down for 9 minutes and 51 seconds).
Event timestamp: 2021-02-08 22:16:37 UTC+0

Here the related mod_ssl errors:

$ ssh root@fabula.wikimedia.it
$ grep 'ssl:emerg' /var/log/httpd/error_log
...
[Mon Feb 08 23:06:07.561250 2021] [ssl:emerg] [pid 32049] AH01910: Oops, no RSA, DSA or ECC server certificate found for 'vda.wikilovesmonuments.it:0'?!
[Mon Feb 08 23:06:07.561265 2021] [ssl:emerg] [pid 32049] AH02312: Fatal error initialising mod_ssl, exiting.
...

At the end the issue was fixed in this way as usual:

  1. find out the problematic SSL VirtualHost and disable the SSL version, until the webserver server is up again
  2. use the plaintext VirtualHost to hammer a new certificate in the problematic SSL VirtualHost(s)
  3. re-enable the problematic SSL VirtualHost
  4. reset the counter of the past days without an SSL issue (yup we had one, we were at 16! whoa!)