Hi all,

Sorry for my delays but the IT company I work for has some issues in another field dedicated to the Ukrainian emergency, where they are volunteering.

But, if everything will be OK I think I will be able to carry out the first raw data and database migration next week.

Hi @valerio.bozzolan,

No worries. I just might have to set Google Analytics OAuth client again then. See the following note https://matomo.org/faq/general/set-up-google-analytics-import/ :

Important note: If you selected External when configuring your Google project, you may have to regularly re-authorize Matomo. Google makes sure authorizations for test users in External projects expire in seven days, at which point your imports will fail and you will have to re-authorize Matomo.

If that's the case, just ping me.

Hello @valerio.bozzolan

Now that Matomo has imported all the stats, can we move forward with the migration ?

Thank you !

I absolutely do not remember our current state. Can you test please this?

https://wmch.fr.dicoado.org/

@Raphoraph @DSwissK Is somebody available this Thursday afternoon for a small Jitsi videocall, to boost this a bit?

Thank you so much!

Hi @valerio.bozzolan, I could be available tomorrow at 14h CEST when my kid will be taking a nap. You can send me the Jitsi link via e-mail.

Thank you !

Yeah let's connect here at 14:00 CEST tomorrow

https://meet.jit.si/DicoAdoTechBoost

I have a problem at 14:00. Is it OK to join at 15:00? Thank you!

https://meet.jit.si/DicoAdoTechBoost

Alright. See you at 15h

I think we have to postpone everything for a week since my health is not 100% OK. Let's stay a bit in my bed XD sorry me.

Yeah no worries. Get better. :)

Here's a beer for when you'll be back healthy again : 🍻

Hello, the fonts and the favicon are OK but...

Can somebody help me in understanding why I'm not able to run mysql or mysqldump (with I assume valid credentials) in the dicoado-legacy server?

I'm talking about my script called wmch-mysql.sh in the home directory of the server.

Thank you so much

Hi, don't know why exactly your script didn't work (perhaps issue with quoting, or password wrongly typed, or idk), but I've put a .my.cnf file with the credentials and now invoking mysql directly works (no need to write the credentials in the arguments).

Just to precise that I fixed and tested your scripts and they work perfectly :-)

You are the number 1.

It works. It seems the full import process takes ~17 minutes. Here time notes:

# start of the dump export and download:
Wed Nov  2 21:55:48 CET 2022

# end of the download:
Wed Nov  2 21:56:05 CET 2022

# end of the MariaDB import:
Wed Nov  2 22:12:22 CET 2022

That's a very nice image by the way :D

BTW I can say that we are still waiting for mailbox catch-all info from our provider, and still need to fix the $ jQuery stuff.

Thank you again @Raphoraph please feel free to update the LocalSettings.php in server wmch-dicoado with the updates you mentioned, but please annotate these changes so we can update /home/www-dicoado/pull.sh accordingly

@valerio.bozzolan: Hi, the Due Date set for this open task passed a while ago.
Could you please either update or reset the Due Date (by clicking Edit Task), or set the status of this task to resolved in case this task is done? Thanks!

@valerio.bozzolan: Hi, the Due Date set for this open task passed a while ago.
Could you please either update or reset the Due Date (by clicking Edit Task), or set the status of this task to resolved in case this task is done? Thanks!

Hi @ValerioBoz-WMCH, can you please update me to what I should now do to make the migration happen, if possible this summer. Thank you ! :)

Probably I have the same question :) I think that WMCH is very happy to be able to offer this server to your community but at this point everything is probably in the hand of the community, who has root access, and control the DNS area.

I do not know if we (me/you) need to update again the pull.sh and test more things again. Maybe it just works.

Feel free to execute that again right now to double-check.

Unfortunately I have not enough experience in the project to verify if everything is OK. I also have not enough credentials to do the switchover, since I do not control the DNS area (but the community does) :(

With some surprise I guess the legacy infrastructure was probably improved some time ago and now this is causing a crash on some pages, probably because of the introduction of the Cargo extension

#0 /var/www/dicoado/fr/httpdocs/w/includes/libs/rdbms/database/DatabaseMysqlBase.php(142): Wikimedia\Rdbms\Database->newExceptionAfterConnectError()
#1 /var/www/dicoado/fr/httpdocs/w/includes/libs/rdbms/database/Database.php(335): Wikimedia\Rdbms\DatabaseMysqlBase->open()
#2 /var/www/dicoado/fr/httpdocs/w/includes/libs/rdbms/database/Database.php(319): Wikimedia\Rdbms\Database->doInitConnection()
#3 /var/www/dicoado/fr/httpdocs/w/includes/libs/rdbms/database/Database.php(444): Wikimedia\Rdbms\Database->initConnection()
#4 /var/www/dicoado/fr/httpdocs/w/extensions/Cargo/includes/CargoUtils.php(87): Wikimedia\Rdbms\Database::factory()
#5 /var/www/dicoado/fr/httpdocs/w/extensions/Cargo/includes/parserfunctions/CargoStore.php(118): CargoUtils::getDB()
#6 /var/www/dicoado/fr/httpdocs/w/includes/parser/Parser.php(3339): CargoStore::run()
#7 /var/www/dicoado/fr/httpdocs/w/includes/parser/Parser.php(3046): Parser->callParserFunction()
#8 /var/www/dicoado/fr/httpdocs/w/includes/parser/PPFrame_Hash.php(263): Parser->braceSubstitution()
#9 /var/www/dicoado/fr/httpdocs/w/extensions/ParserFunctions/includes/ParserFunctions.php(146): PPFrame_Hash->expand()
#10 /var/www/dicoado/fr/httpdocs/w/includes/parser/Parser.php(3339): MediaWiki\Extensions\ParserFunctions\ParserFunctions::ifeq()
#11 /var/www/dicoado/fr/httpdocs/w/includes/parser/Parser.php(3046): Parser->callParserFunction()
#12 /var/www/dicoado/fr/httpdocs/w/includes/parser/PPFrame_Hash.php(263): Parser->braceSubstitution()
#13 /var/www/dicoado/fr/httpdocs/w/includes/parser/Parser.php(3224): PPFrame_Hash->expand()
#14 /var/www/dicoado/fr/httpdocs/w/includes/parser/PPFrame_Hash.php(263): Parser->braceSubstitution()
#15 /var/www/dicoado/fr/httpdocs/w/extensions/ParserFunctions/includes/ParserFunctions.php(121): PPFrame_Hash->expand()
#16 /var/www/dicoado/fr/httpdocs/w/includes/parser/Parser.php(3339): MediaWiki\Extensions\ParserFunctions\ParserFunctions::if()
#17 /var/www/dicoado/fr/httpdocs/w/includes/parser/Parser.php(3046): Parser->callParserFunction()
#18 /var/www/dicoado/fr/httpdocs/w/includes/parser/PPFrame_Hash.php(263): Parser->braceSubstitution()
#19 /var/www/dicoado/fr/httpdocs/w/extensions/ParserFunctions/includes/ParserFunctions.php(146): PPFrame_Hash->expand()
#20 /var/www/dicoado/fr/httpdocs/w/includes/parser/Parser.php(3339): MediaWiki\Extensions\ParserFunctions\ParserFunctions::ifeq()
#21 /var/www/dicoado/fr/httpdocs/w/includes/parser/Parser.php(3046): Parser->callParserFunction()
#22 /var/www/dicoado/fr/httpdocs/w/includes/parser/PPFrame_Hash.php(263): Parser->braceSubstitution()
#23 /var/www/dicoado/fr/httpdocs/w/includes/parser/Parser.php(3224): PPFrame_Hash->expand()
#24 /var/www/dicoado/fr/httpdocs/w/includes/parser/PPFrame_Hash.php(263): Parser->braceSubstitution()
#25 /var/www/dicoado/fr/httpdocs/w/includes/parser/Parser.php(2886): PPFrame_Hash->expand()
#26 /var/www/dicoado/fr/httpdocs/w/includes/parser/Parser.php(1557): Parser->replaceVariables()
#27 /var/www/dicoado/fr/httpdocs/w/includes/parser/Parser.php(652): Parser->internalParse()
#28 /var/www/dicoado/fr/httpdocs/w/includes/content/WikitextContent.php(374): Parser->parse()
#29 /var/www/dicoado/fr/httpdocs/w/includes/content/AbstractContent.php(590): WikitextContent->fillParserOutput()
#30 /var/www/dicoado/fr/httpdocs/w/includes/Revision/RenderedRevision.php(263): AbstractContent->getParserOutput()
#31 /var/www/dicoado/fr/httpdocs/w/includes/Revision/RenderedRevision.php(235): MediaWiki\Revision\RenderedRevision->getSlotParserOutputUncached()
#32 /var/www/dicoado/fr/httpdocs/w/includes/Revision/RevisionRenderer.php(215): MediaWiki\Revision\RenderedRevision->getSlotParserOutput()
#33 /var/www/dicoado/fr/httpdocs/w/includes/Revision/RevisionRenderer.php(152): MediaWiki\Revision\RevisionRenderer->combineSlotOutput()
#34 [internal function]: MediaWiki\Revision\RevisionRenderer->MediaWiki\Revision\{closure}()
#35 /var/www/dicoado/fr/httpdocs/w/includes/Revision/RenderedRevision.php(197): call_user_func()
#36 /var/www/dicoado/fr/httpdocs/w/includes/poolcounter/PoolWorkArticleView.php(216): MediaWiki\Revision\RenderedRevision->getRevisionParserOutput()
#37 /var/www/dicoado/fr/httpdocs/w/includes/poolcounter/PoolCounterWork.php(162): PoolWorkArticleView->doWork()
#38 /var/www/dicoado/fr/httpdocs/w/includes/page/Article.php(810): PoolCounterWork->execute()
#39 /var/www/dicoado/fr/httpdocs/w/includes/actions/ViewAction.php(80): Article->view()
#40 /var/www/dicoado/fr/httpdocs/w/includes/MediaWiki.php(531): ViewAction->show()
#41 /var/www/dicoado/fr/httpdocs/w/includes/MediaWiki.php(313): MediaWiki->performAction()
#42 /var/www/dicoado/fr/httpdocs/w/includes/MediaWiki.php(947): MediaWiki->performRequest()
#43 /var/www/dicoado/fr/httpdocs/w/includes/MediaWiki.php(547): MediaWiki->main()
#44 /var/www/dicoado/fr/httpdocs/w/index.php(53): MediaWiki->run()
#45 /var/www/dicoado/fr/httpdocs/w/index.php(46): wfIndexMain()
#46 {main}

So I also imported database nxxs_dicoado_cargo_fr. Now it works.

@DSwissK thanks for your support. Everything was fixed as far as I can see in the import process. It's better to don't do this migration today for multiple reasons (for example, the DNS TTL was too big. Today we will have more issues).

The final migration will happen next week the 2023-09-07 · 09:00 CEST

The final migration will happen next week the 2023-09-07 · 14:00 CEST (reason: requested by DSwissK)

In that moment I will set read only mode in the legacy wiki.

Because of this, I disabled maintenance mode.

I'm starting the process right now.

First, the bad news:

Good news: the migration in general worked. So we can re-try this as soon as the DNS will start working again and it will only take 20 minutes thanks to our migration script.

I remove the read-only mode from the legacy server.

Good news here

We were able to find the root cause. The Anycast option in the DNS area of Infomaniak was enabled only for the .org and it seems it was a kind of super-aggressive-mode. After disabling it, the DNS area started propagating correctly like a little cute horse free to express freedom (?).

After that, I've re-executed all the bells and whistles (e.g. maintenance, pull.sh etc.) and... it worked.

I've issued let's encrypt certificates for dicoado.org and www.dicoado.org (probably still missing fr.dicoado.org - we are still using the original ones that I imported previously)

I tried also to simplify a bit the homepage that was relying on this esoteric HTML tag:

<base href="https://dicoado.org/mainpage/" target="_blank">

Dropped, and moved images.

For interesting reasons, everything works, even if I've done all of this this from a train to Venice, to reach the End Summer Camp conference. Nice.

Before Monday at 10:00 the above checklist will be verified again and, probably, this Task will be formally marked as resolved \o/

Alright, there is a nasty bug the we just discovered : since the migration, we cannot edit any word using our formular.

Thanks for the work so far Valerian!
However, for this one, emphasis mine:

I've issued let's encrypt certificates for dicoado.org and www.dicoado.org (probably still missing fr.dicoado.org - we are still using the original ones that I imported previously)

It would have been better to renew all the certificates (perhaps using wildcard?) :D.

The bug that DSwissK reported was apparently due to the certificate of fr.dicoado.org being misconfigured somehow. While the browser apparently finds his way around, less "smart" tools like wget, curl, or the internal HTTP system used by VEForAll fails to properly verify the certificate, hence dropping the connection. This makes the communication with Parsoid fail and finally making the edit form unusable. This post in stackexchange talks about this kind of problem.

In order to fix it I've generated a new certificate for fr.dicoado.org. I write down the steps here since I might not have done everything like you.
Used this link for reference (only for the usage part since certbot was already there).

sudo certbot certonly --apache
# Selected fr.dicoado.org as the domain I want to use, certbot then verified ownership through http method (and cleaned it itself AFAIK).
sudo certbot renew --dry-run
# Everything seemed to work fine.

I then mimiced the configuration you used for the other domains in order to change the file org-dicoado-fr-ssl.conf. Feel free to check it.
Note that contrary to the other certificates, I used the method apache and not webroot. It still worked probably because everything was already in place so that certbot finds his way. Feel free to change the parameters for renewal.

Other remarks I would like to make :-)

• The server is a bit of a mess for the moment :/ There are different occurences of a MediaWiki installations, configuration files are stored in really heterogenous places, and there are some test files that lies around. This makes maintenance difficult, and we could gain from cleaning a little^^
• I've moved the debug configurations you added in LocalSettings.php to their proper place (sooner in the file). This has made me go crazy for a moment because I was setting wgDebugLogFile where I was used to do so without realizing I was being overriden by the config at the end of the file :'-). I've put the configurations behind a barrier to avoid leaks and also disabled default logging: this is far too verbose, only a few days of being on and the output file was 1.7 GB. Also, it stores a lot of private informations. This option is better enabled only when trouble shooting a specific bug in my opinion, I therefore disabled it and removed the fat debug file.
• I think we could gain in simplicity by using wildcard for certificates but also for virtualhosts and domain (have a configuration for *dicoado.org instead of fr and www.fr). However I understand that it is harder to put in place!
• We could take more advantage of having now our own server (with root access). For example you've made a TODO for moving .htaccess configuration into the real apache configuration. We can also strengthen the filesystem permission settings in order to reduce attack surface (I've done this for the private folder but this can be done everywhere). Still about reducing attack surface, the apache config could be simplified and the enabled options reduced. This is something I can also help to do.

Again thanks for your time!

Indeed lot of things to be done

Thanks for that certificate, indeed certbot is amazing.

I think we could gain in simplicity by using wildcard for certificates .. However I understand that it is harder to put in place

Eheh I think we already have the answer. Wildcard certificates requires also to update a DNS record every single time they are supposed to be renewed. This is indeed an additional layer of complexity for any DNS server and involves unpredictable work / collaboration from the DNS service provider itself. At the moment, for an unrelated project in Italian Linux Society, we are dealing with wild certificates (not for choice) since 3 years, and nobody has found any "keep-it-simple-and-stupid" solution, so at the moment I do it manually for them (!) since PowerDNS is not compatible with our server and we have not any intention to migrate to Cloudflare and other time-related reasons :( But if you have tips, please share! https://gitpull.it/T96

Without wildcard certificates, I also do not recommend the creation of "big" certificates with lot of sub-domains. At the moment we have 6 certificates:

dicoado.ch (with www)
dicoado.com  (with www)
dicoado.org  (with www)
fr.dicoado.ch  (with www)
fr.dicoado.com  (with www)
fr.dicoado.org  (with www)

This is a feature at the moment: the primary reason is: Let's Encrypt allows to generate a very big number of certificates, and you can create new one running certbot and it just works and it's relatively super-easy and lovely, at no cost, and their renewals are already automated "forever" (until the world crash) and it works without any user interaction.

Indeed we can cleanup. But, having modular VirtualHosts with smaller certificates is already a cleanup in my opinion, so we gained no side-effects on other virtualhosts. At least until wildcards are so painfully to be setup (that would be better)...

Thanks for that certificate, indeed certbot is amazing.

Can confirm 😄

(For info I've removed a asd.txt file you added, probably to test)

I didn't realize wildcard certificates were much harder than that, sorry! So yes, it wouldn't be efficient to use this. If I come up with a magical idea I will let you know 😅
In our case, perhaps still the VirutalHost configuration scheme could be easier to understand by avoiding multiplications of different files to look at (still having virtual host directives but regrouped), but I'm not really sure, and it's not really important.

I recognize that I am still in the "discovering the new server" process so I need to adapt, sorry about this.

Yeah yeah feel free to group things

Don't tell anyone that I love to create little cute VirtualHosts here and there.

Example in WMIT (conducted in volunteer time)

https://gitlab.wikimedia.org/repos/wikimedia-it/wmit-infrastructure/-/tree/main/servers/intreccio/conf/apache2/sites-available

For info I've removed a asd.txt file you added, probably to test)

Wait WHAT you removed the asd? O.o

That was a super-important mandatory file. Example (again, don't tell anyone):

https://members.wikimedia.ch/asd

@ValerioBoz-WMCH: Hi, the Due Date set for this open task passed a while ago.
Could you please either update or reset the Due Date (by clicking Edit Task), or set the status of this task to resolved in case this task is done? Thanks!