Make beta German Wiktionary
Closed, ResolvedPublic

Description

Per our discussion in WMDE weekly meeting. We need it to test Cognate extension.

Restricted Application added a project: User-Ladsgroup. · View Herald TranscriptNov 15 2016, 4:01 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript

Change 321673 had a related patch set uploaded (by Ladsgroup):
Add German Wiktionary in beta

https://gerrit.wikimedia.org/r/321673

Change 321673 merged by jenkins-bot:
Add German Wiktionary in beta

https://gerrit.wikimedia.org/r/321673

There are instructions for adding new wikis: https://wikitech.wikimedia.org/wiki/Nova_Resource:Deployment-prep/Add_a_wiki .

There are at least two issues I see right now:

  1. The TLS certificate (P4445)
  2. The External Store table (it doesn't exist so edits fail)

It doesn't look like addWiki.php handles 1, but it does 2. I'll go through and try to finish this setup now.

Mattflaschen-WMF added a comment.EditedNov 15 2016, 11:34 PM

Ran:

mwscript extensions/WikimediaMaintenance/filebackend/setZoneAccess.php dewiktionary --backend=local-multiwrite

(Swift)

Mattflaschen-WMF added a comment.EditedNov 15 2016, 11:43 PM

Thanks, @Krenair. Is the TLS certificate fully automatic except for editing the wiki page? (apparently yes since it's started working)

(Edited)

Krenair added a comment.EditedNov 15 2016, 11:48 PM

Thanks, @Krenair. Is the TLS certificate fully automatic except for editing the wiki page? (apparently yes since it's started working)

(Edited)

your first edit was correct, the second was not (I've reverted it). puppet on -cache-text04 has taken care of applying it and the reissuance of the cert from LE

Change 321810 had a related patch set uploaded (by Mattflaschen):
Add Flow External Store to addWiki.php

https://gerrit.wikimedia.org/r/321810

your first edit was correct, the second was not (I've reverted it). puppet on -cache-text04 has taken care of applying it and the reissuance of the cert from LE

Thanks, documented.

Change 321816 had a related patch set uploaded (by Mattflaschen):
Add dewiktionary for Beta Cluster

https://gerrit.wikimedia.org/r/321816

Change 321817 had a related patch set uploaded (by Mattflaschen):
Add dewiktionary to RESTBase on Beta Cluster

https://gerrit.wikimedia.org/r/321817

All the steps are done, except:

  • Merge the last two changes above (Parsoid and RESTBase, they are automatically deployed)
  • After https://gerrit.wikimedia.org/r/321810 is reviewed, we need to drop the dewiktionary database and re-run addWiki.php to do all the steps (including re-creating that database, and also the External Store issue)

Change 321816 merged by jenkins-bot:
Add dewiktionary for Beta Cluster

https://gerrit.wikimedia.org/r/321816

So this happened:

[ERROR] Slave SQL: Error 'Can't drop database 'dewiktionary'; database doesn't exist' on query. Default database: 'dewiktionary'. Query: 'drop database dewiktionary', Gtid 0-172234526-802746, Internal MariaDB error code: 1008
[ERROR] Error running query, slave SQL thread aborted. Fix the problem, and restart the slave SQL thread with "SLAVE START". We stopped at log 'deployment-db03-bin.000009' position 592967822
hashar triaged this task as Normal priority.Nov 16 2016, 9:04 AM
hashar moved this task from To Triage to Backlog on the Beta-Cluster-Infrastructure board.

Mentioned in SAL (#wikimedia-releng) [2016-11-16T12:24:28Z] <hashar> beta: created dewiktionary table on the Database slave. Restarted replication with START SLAVE; T150834 T150764

hashar added a subscriber: hashar.Nov 16 2016, 12:27 PM

From T150834:

That has been caused by the addition of the German Wiktionary ( T150764 ). The MySQL master/slave replication got broken because the dewiktionary database was not created on the slave and "drop database dewiktionary;" failed to replicate.

I have manually created the database on the slave, issued START SLAVE and replication caught up from there.

update.php works now! https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/12839/console

And the job fails:

$ mwscript update.php --wiki=dewiktionary  --quick
#!/usr/bin/env php
MediaWiki 1.29.0-alpha Updater

Your composer.lock file is up to date with current dependencies!

dewiktionary is not on the master database. So gotta remove it from the dblist. Eg I am going to revert the change.

Change 321866 had a related patch set uploaded (by Hashar):
Revert "Add German Wiktionary in beta"

https://gerrit.wikimedia.org/r/321866

Change 321866 merged by jenkins-bot:
Revert "Add German Wiktionary in beta"

https://gerrit.wikimedia.org/r/321866

To unblock T150763, the dewiktionary database is no more on the MySQL master and slave. I also removed it from the dblist and wikiversions.json. The addWiki.php will have to be done again and https://gerrit.wikimedia.org/r/321673 cherry picked again.

Change 321810 merged by jenkins-bot:
Add Flow External Store to addWiki.php

https://gerrit.wikimedia.org/r/321810

That has been caused by the addition of the German Wiktionary ( T150764 ). The MySQL master/slave replication got broken because the dewiktionary database was not created on the slave and "drop database dewiktionary;" failed to replicate.

I'm just wondering how that happened (only creating dewiktionary on the master), and in general how the tables were created before (because it was apparently not addWiki.php).

I'll follow up later today.

greg added a subscriber: greg.Nov 16 2016, 8:02 PM

That has been caused by the addition of the German Wiktionary ( T150764 ). The MySQL master/slave replication got broken because the dewiktionary database was not created on the slave and "drop database dewiktionary;" failed to replicate.

I'm just wondering how that happened (only creating dewiktionary on the master), and in general how the tables were created before (because it was apparently not addWiki.php).

@Ladsgroup ^^

Change 322010 had a related patch set uploaded (by Mattflaschen):
Add German Wiktionary in beta (2nd try)

https://gerrit.wikimedia.org/r/322010

Change 322010 merged by jenkins-bot:
Add German Wiktionary in beta (2nd try)

https://gerrit.wikimedia.org/r/322010

addWiki failed saying the database existed already (?), so now I have to delete it again.

I think addWiki.php somehow creates it twice, not sure where.

Change 322017 had a related patch set uploaded (by Mattflaschen):
Take into account that Echo is not always on extension1

https://gerrit.wikimedia.org/r/322017

Change 322017 merged by jenkins-bot:
Take into account that Echo is not always on extension1

https://gerrit.wikimedia.org/r/322017

Change 322022 had a related patch set uploaded (by Mattflaschen):
Handle $wgCirrusSearchWriteClusters being null, add missing global

https://gerrit.wikimedia.org/r/322022

Change 322022 merged by jenkins-bot:
Handle $wgCirrusSearchWriteClusters being null, add missing global

https://gerrit.wikimedia.org/r/322022

Mattflaschen-WMF closed this task as Resolved.Nov 17 2016, 12:52 AM

Done.

There's a thing with wmgExtraLanguageNames, but doesn't seem to break anything. No errors from addWiki.php itself:

P4457

Mattflaschen-WMF reopened this task as Open.Nov 17 2016, 12:56 AM

EnableFlow isn't working, hopefully just due to https://gerrit.wikimedia.org/r/#/c/321817/ not being merged yet.

Actually it was https://gerrit.wikimedia.org/r/#/c/322033/ .

It still doesn't work, but Petr told me Parsoid is known to not be working on Beta (probably T150897).

I get the same error on en.wikipedia Beta as well.

So I'll just leave this open for the RESTBase to be merged, but other than that, it's probably just T150897.

Ladsgroup moved this task from needs review to done on the WMDE-TLA-Team board.Nov 17 2016, 9:16 AM
Ladsgroup moved this task from Incoming to Done on the User-Ladsgroup board.
Ladsgroup moved this task from Proposed to Done on the Wikidata-Sprint board.Nov 18 2016, 10:32 AM
hashar removed a subscriber: hashar.Nov 18 2016, 12:14 PM

Change 321817 merged by Dzahn:
Add dewiktionary to RESTBase on Beta Cluster

https://gerrit.wikimedia.org/r/321817

Micru updated the task description. (Show Details)Jan 6 2017, 12:45 PM
WMDE-leszek closed this task as Resolved.Mar 7 2017, 3:47 PM
WMDE-leszek added a subscriber: WMDE-leszek.