MWMultiVersion.php Fatal error: no version entry for `#` (was: special pages has not been updated since November 2019 in jawiki and several other projects)
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	altocool
	Jan 24 2020, 1:15 PM

Description

Some of the special pages has not been updated since around November 2019 in jawiki. Is there any reason this cannot be updated?

https://ja.wikipedia.org/wiki/%E7%89%B9%E5%88%A5:%E7%89%B9%E5%88%A5%E3%83%9A%E3%83%BC%E3%82%B8%E4%B8%80%E8%A6%A7

特別:更新されていないページ (Special:AncientPages) - Last update 01:02, 22 Nov 2019.
特別:編集履歴の少ないページ (Special:FewestRevisions) - Last update 01:52, 22 Nov 2019.
特別:有効なページへのリンクがないページ (Special:LonelyPages) - Last update 01:16, 22 Nov 2019.
特別:編集履歴の多いページ (Special:MostRevisions) - Last update 01:55, 22 Nov 2019.
特別:被リンクの多いページ (Special:MostTranscludedPages) - Last update 04:58, 22 Nov 2019.

Related Objects

Mentioned In: T247716: Evaluate all scripts that use dblists to make sure they'll deal with comments
T247706: Update of special wanted categories at nowiki
T246717: Refresh https://nl.wikipedia.org/wiki/Speciaal:MeesteVersies
Mentioned Here: rOMWCfa9812e31415: Variant configuration: Generate dblists from YAML
rOPUP1606b13af84e: make mwscriptwikiset respect comments set in dblists

Event Timeline

altocool created this task.Jan 24 2020, 1:15 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJan 24 2020, 1:15 PM

Mainframe98 added a project: Wikimedia-maintenance-script-run.Jan 24 2020, 1:35 PM

I think the cron for this group should have run aprox on the 10th & 24th of the month

from /var/log/mediawiki/updateSpecialPages.log on mwmaint1001:

17371 jawiki
17372 -----------------------------------------------------------------
17373 jawiki:  Statistics                     [callback] completed in 22.47s
17374 jawiki:  Ancientpages                   [QueryPage] disabled
17375 jawiki:  BrokenRedirects                [QueryPage] got 152 rows in 10.87s
17376 jawiki:  Deadendpages                   [QueryPage] disabled
17377 jawiki:  DoubleRedirects                [QueryPage] got 23 rows in 5.90s
17378 jawiki:  FileDuplicateSearch            [QueryPage] cheap, skipped
17379 jawiki:  ListDuplicatedFiles            [QueryPage] got 3 rows in 0.62s
17380 jawiki:  LinkSearch                     [QueryPage] cheap, skipped
17381 jawiki:  Listredirects                  [QueryPage] got 5000 rows in 0.24s
17382 jawiki:  Lonelypages                    [QueryPage] got 2868 rows in 1m 32.43s
17383 jawiki:  Longpages                      [QueryPage] cheap, skipped
17384 jawiki:  MediaStatistics                [QueryPage] got 8 rows in 0.21s
17385 jawiki:  MIMEsearch                     [QueryPage] cheap, skipped
17386 jawiki:  Mostcategories                 [QueryPage] got 5000 rows in 1m 14.49s
17387 jawiki:  Mostimages                     [QueryPage] got 5000 rows in 10.78s
17388 jawiki:  Mostinterwikis                 [QueryPage] got 5000 rows in 55.21s
17389 jawiki:  Mostlinkedcategories           [QueryPage] cheap, skipped
17390 jawiki:  Mostlinkedtemplates            [QueryPage] got 5000 rows in 51.58s
17391 jawiki:  Mostlinked                     [QueryPage] disabled
17392 jawiki:  Mostrevisions                  [QueryPage] disabled
17393 jawiki:  Fewestrevisions                [QueryPage] disabled
17394 jawiki:  Shortpages                     [QueryPage] cheap, skipped
17395 jawiki:  Uncategorizedcategories        [QueryPage] got 2 rows in 4.63s
17396 jawiki:  Uncategorizedpages             [QueryPage] got 47 rows in 13.83s
17397 jawiki:  Uncategorizedimages            [QueryPage] got 3 rows in 0.74s
17398 jawiki:  Uncategorizedtemplates         [QueryPage] got 777 rows in 0.58s
17399 jawiki:  Unusedcategories               [QueryPage] got 4943 rows in 15.92s
17400 jawiki:  Unusedimages                   [QueryPage] got 5000 rows in 0.32s
17401 jawiki:  Wantedcategories               [QueryPage] got 5000 rows in 14.35s
17402 jawiki:  Wantedfiles                    [QueryPage] got 5000 rows in 27.31s
17403 jawiki:  Wantedpages                    [QueryPage] disabled
17404 jawiki:  Wantedtemplates                [QueryPage] got 5000 rows in 13.33s
17405 jawiki:  Unwatchedpages                 [QueryPage] got 5000 rows in 0.54s
17406 jawiki:  Unusedtemplates                [QueryPage] got 5000 rows in 0.58s
17407 jawiki:  Withoutinterwiki               [QueryPage] got 5000 rows in 0.42s
17408 jawiki:  UnconnectedPages               [QueryPage] cheap, skipped
17409 jawiki:  GadgetUsage                    [QueryPage] got 40 rows in 2m 10.96s
17410 jawiki:  MostGloballyLinkedFiles        [QueryPage] got 0 rows in 0.00s
17411 jawiki:  GloballyWantedFiles            [QueryPage] got 0 rows in 0.00s
17412 jawiki:  DisambiguationPages            [QueryPage] cheap, skipped
17413 jawiki:  DisambiguationPageLinks        [QueryPage] got 5000 rows in 4.14s
17414 -----------------------------------------------------------------

So some of the things you mention are listed as "disabled" but some are not.

In T243599#5831093, @Peachey88 wrote:

I think the cron for this group should have run aprox on the 10th & 24th of the month

Jan 22 22:14 is when updateSpecialPages.log has been written the last time

@Dzahn I guess the question then becomes - why were they disabled?

jijiki triaged this task as Medium priority.Jan 28 2020, 12:56 PM

Looking at what is disabled in general, i see this is not specific to Japanese Wikipedia but globally these are disabled across languages:

Ancientpages
Deadendpages
Mostlinked
Mostrevisions
Fewestrevisions
Wantedpages

Yeah, looks like this didn't run in the four times expected since (two in December, two in January).

Disabling is not at the puppet / cronjob level. These are "sub-jobs" of the single "update_special_pages" job. The script itself is in mediawiki/core, the configuration for it is in mediawiki-config/wmf-config/InitialiseSettings.php.

13263 # Disable all the query pages that take more than about 15 minutes to update
13264 # wgDisableQueryPageUpdate @{
13265 'wgDisableQueryPageUpdate' => [
13266         'default' => [
13267                 'Ancientpages' => 'periodical',
13268                 'Deadendpages' => 'periodical',
13269                 'Fewestrevisions' => 'periodical',
13270                 'Mostlinked' => 'periodical',
13271                 'Mostrevisions' => 'periodical',
13272                 'Wantedpages' => 'periodical',
13273         ],

So the answer to why they are disabled seems to be "because they take more than 15 minutes to update".

git blame for these lines brings us to:

https://gerrit.wikimedia.org/r/c/operations/mediawiki-config/+/541358

Dzahn added a subscriber: Umherirrender.Jan 28 2020, 6:45 PM

In T243599#5836900, @Dzahn wrote:

git blame for these lines brings us to:

https://gerrit.wikimedia.org/r/c/operations/mediawiki-config/+/541358

Yeah, does whatever in puppet is parsing config not understand the new value?

@Umherirrender Looking at the change above and this ticket, do you know more about why and when these jobs were disabled?

In T243599#5836920, @Dzahn wrote:

@Umherirrender Looking at the change above and this ticket, do you know more about why and when these jobs were disabled?

The jobs have been disabled for over a decade. This is about the cron replacement for the jobs not running, since the config update.

In T243599#5836911, @Jdforrester-WMF wrote:

Yeah, does whatever in puppet is parsing config not understand the new value?

I don't think anything in puppet parses config. From puppet's point of view there is just this:

20     $ensure = mediawiki::state('primary_dc') ? {
21         $::site => 'present',
22         default => 'absent',

and then all the maintenance crons are enabled or disabled depending on "am i in the primary datacenter".

Additionally "update_special_pages" is a single cron job at puppet level and there is no difference between specific special pages.

In puppet is mediawiki::maintenance::updatequerypages::ancientpages which is running the update with the override option to get the special page updated.

It is defined with monthday => [8, 22],

And called from mediawiki::maintenance::updatequerypages

mediawiki::maintenance::updatequerypages::ancientpages { ['s1@11', 's2@12', 's3@13', 's4@14', 's5@15', 's6@16', 's7@17', 's8@18', 'wikitech@19']: }

itself called

# (T17434) Periodical run of currently disabled special pages
class { 'mediawiki::maintenance::updatequerypages': ensure => $ensure }

jawiki lives on s6, which is part of the config. But the update also not running on dewiki, so it seems more than just this maintenance is not running
Please have a look at the logs of that cron job.

The config for DisableQueryPageUpdate is only to show on the special page, the cron jobs must be set up to get the "half-monthly"

# Disable all the query pages that take more than about 15 minutes to update
# Specifiy a "run mode" when a cron job is set up for the special page and
# also create a message for the "run mode" in WikimediaMessages extension with
# the message key 'querypage-updates-<run mode>'

from [mwmaint1002:/var/log/mediawiki/updateSpecialPages/s6@16-AncientPages.log] $

  1 -------------------------------------
  2 #
  3 -------------------------------------
  4 no version entry for `#`.
  5 Fatal error: no version entry for `#`.
  6  in /srv/mediawiki/multiversion/MWMultiVersion.php on line 423
~              

And this Fatal error shows up in all the other shards as well for the same crons.

revi added a subscriber: revi.Jan 29 2020, 12:54 PM

Agabi10 added a subscriber: Agabi10.Jan 29 2020, 1:06 PM

In T243599#5837194, @Dzahn wrote:

from [mwmaint1002:/var/log/mediawiki/updateSpecialPages/s6@16-AncientPages.log] $

  1 -------------------------------------
  2 #
  3 -------------------------------------
  4 no version entry for `#`.
  5 Fatal error: no version entry for `#`.
  6  in /srv/mediawiki/multiversion/MWMultiVersion.php on line 423
~              

And this Fatal error shows up in all the other shards as well for the same crons.

There is nothing I can help:

	/**
	 * Get the version as specified in a php file located
	 * at /srv/mediawiki/wikiversions.php.
	 * Result is of the form "php-X.XX" or "php-trunk".
	 * @return String the version directory for this wiki
	 */
	public function getVersion() {
		$this->loadVersionInfo();
		if ( $this->version === false ) {
			self::error( "no version entry for `{$this->db}`.\n" );
		}
		return $this->version;
	}

I can confirm that this is impacting many projects. E.g. https://en.wikipedia.org/wiki/Special:WantedPages and https://www.wikidata.org/wiki/Special:WantedPages and https://species.wikimedia.org/wiki/Special:WantedPages haven't been updated since 2019-11-25 and the latter two claim that they will be updated every two weeks or so (en.wp has a huge warning that it's rarely updated).

Koavf added a comment.Feb 13 2020, 10:03 PM

This comment was removed by Koavf.

Pigsonthewing added a subscriber: Pigsonthewing.Feb 27 2020, 1:53 PM

Ciell mentioned this in T246717: Refresh https://nl.wikipedia.org/wiki/Speciaal:MeesteVersies.Mar 2 2020, 9:16 PM

Mbch331 merged a task: T246717: Refresh https://nl.wikipedia.org/wiki/Speciaal:MeesteVersies.Mar 2 2020, 9:18 PM

Mbch331 added a subscriber: Ciell.

Mbch331 added a subscriber: Mbch331.

Hi all,

Because Dutch Wikipedia will be hitting a mile stone somewhere this month, I'd like to present some nice details of the project to the press, but our Special: pages are also effected by this.
Will this be solved any time soon maybe?

This is not urgent/break now but it's pretty important for maintenance on several wikis and has evidently been broken for four or five months.

Dzahn renamed this task from special pages has not been updated since November 2019 in jawiki and several other projects to MWMultiVersion.php Fatal error: no version entry for `#` (was: special pages has not been updated since November 2019 in jawiki and several other projects).Mar 3 2020, 2:14 AM

Dzahn added a project: Platform Engineering.

1606b13af84e237f2aeb68e86dc46998d6f9dd62 should've fixed this...

Putting this back at Medium. The underlying issue is fixed, and the scripts will work on the next runs

This was only fixed on the 25th Feb, so any maintenance script that hasn't run since, won't have run successfully

https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/574726/

Mentioned in SAL (#wikimedia-operations) [2020-03-03T02:39:26Z] <mutante> manually running updateSpecialPages.php maintenance cron on s8 for AncientPages to confirm it was fixed by gerrit:574726 a few days ago (T243599)

Noting this was broken by fa9812e31415e4afb415114fe7c0f31520199b30, https://gerrit.wikimedia.org/r/#/c/operations/mediawiki-config/+/545411/ late November 2019

Restricted Application added a project: User-Urbanecm. · View Herald TranscriptMar 3 2020, 2:41 AM

Mentioned in SAL (#wikimedia-operations) [2020-03-03T02:57:10Z] <mutante> manually running "Ancientpages" cron on s3 (T243599)

Reedy mentioned this in T247706: Update of special wanted categories at nowiki.Mar 15 2020, 6:29 PM

Reedy mentioned this in T247716: Evaluate all scripts that use dblists to make sure they'll deal with comments.Mar 15 2020, 9:41 PM

MWMultiVersion.php Fatal error: no version entry for `#` (was: special pages has not been updated since November 2019 in jawiki and several other projects)Closed, ResolvedPublicActions

Description

Related Objects

Event Timeline

MWMultiVersion.php Fatal error: no version entry for `#` (was: special pages has not been updated since November 2019 in jawiki and several other projects)
Closed, ResolvedPublic
Actions