Page MenuHomePhabricator

Missing content_model rows after importing templates and modules
Closed, InvalidPublicBUG REPORT

Description

Goal: To export citation templates from Wikipedia and successfully import them into a non-WMF wiki

Found instructions: https://www.mediawiki.org/wiki/VisualEditor/Citation_tool cursory instructions on how to import citation templates

Info: The Mediawiki is an "empty" MW 1.35 set up just for testing changes. The MW was not installed via git clone and has been upgraded to the current version.

Steps to Reproduce:
Installed and enabled these extensions from git:

mw:Extension:TemplateData
mw:Extension:TemplateStyles

Ran composer in extensions/TemplateData and extensions/TemplateStyles: composer update --no-dev
Ran php maintenance/update.php and php maintenance/runJobs.php

Export templates needed with Special:Export, choose only latest versions, not full histories and ticked the box "include templates" and asked it to export

w:Template:Cite web
w:Template:Cite news
w:Template:Cite journal
w:Template:Cite book
w:Template:Cite AV media
w:Template:Citation

Did try to alter the database and insert the following values to content_models as per what Vulpix had in his database.

INSERT INTO content_models VALUES (2, 'css');
INSERT INTO content_models VALUES (3, 'javascript');
INSERT INTO content_models VALUES (4, 'Scribunto');
INSERT INTO content_models VALUES (5, 'sanitized.css');
commit;

Import the templates with Special:Import finishes fine, but the citation templates are not working

Irc snippets from helpful people in the mediawiki-channel on freenode on 2020-11-19

[20:52] <Vulpix> This would be the case if scribunto or TemplateStyles relies on update.php to populate the content_models table instead of "lazy-populating" it
[20:53] <Vulpix> Still, requesting a greater model_id than the greatest one present on the database is odd
[20:55] <onejukeboksi> Vulpix: I can confirm that it is trying to access an id that doesn't exist, because now the Internal error dump complains about 'id = 6' not being there. I just inserted values for 2,3,4 and 5
[20:58] <Vulpix> or maybe the contents of the content_models get cached on a service/memory/class, the new model gets inserted, but the internal cache doesn't get populated. When it tries to access the new created model from the cache, it's not found, crashes, and roll-backs the entire transaction
[20:58] <Vulpix> that would make more sense

Actual Results:

Any page (?) with citation templates an internal error:

''[5fbc8a084c204cb4609b0020] /w/index.php?title=Testing&action=submit MediaWiki\Storage\NameTableAccessException from line 43 of /var/www/kunsolan.fi/testing/wiki/w/includes/Storage/NameTableAccessException.php: Failed to access name from content_models using id = 6''

If I did not touch the content_model the error was seemingly the same, except complaining about inability to access record with id=2

Expected Results:

Successfully importing the templates

Log of export-imported articles

On 2020-11-19 the following templates were exported from Wikipedia to the private testwiki

w:Template:Cite web
w:Template:Cite news
w:Template:Cite journal
w:Template:Cite book
w:Template:Cite AV media
w:Template:Citation

Selecting "Include templates" in the Special:Export lead to the following additional templates and modules to be exported

w:Template:Cite web
w:Template:Cite news
w:Template:Cite journal
w:Template:Cite book
w:Template:Cite AV media
w:Template:Citation
w:Template:!xt
w:Template:=
w:Template:AWB standard installation
w:Template:Anchor
w:Template:Category link
w:Template:Citation Style 1
w:Template:Citation Style documentation
w:Template:Citation Style documentation/agency
w:Template:Citation Style documentation/author
w:Template:Citation Style documentation/auto-date-format
w:Template:Citation Style documentation/coins
w:Template:Citation Style documentation/cs1
w:Template:Citation Style documentation/date
w:Template:Citation Style documentation/deprecated
w:Template:Citation Style documentation/display
w:Template:Citation Style documentation/editor
w:Template:Citation Style documentation/id1
w:Template:Citation Style documentation/id2
w:Template:Citation Style documentation/journal
w:Template:Citation Style documentation/language
w:Template:Citation Style documentation/lay
w:Template:Citation Style documentation/lead
w:Template:Citation Style documentation/lua
w:Template:Citation Style documentation/pages
w:Template:Citation Style documentation/publisher
w:Template:Citation Style documentation/quote
w:Template:Citation Style documentation/ref
w:Template:Citation Style documentation/registration
w:Template:Citation Style documentation/required
w:Template:Citation Style documentation/sep period
w:Template:Citation Style documentation/series
w:Template:Citation Style documentation/syntax
w:Template:Citation Style documentation/type
w:Template:Citation Style documentation/url
w:Template:Citation Style documentation/usage
w:Template:Citation Style documentation/usage common
w:Template:Citation Style documentation/usage full
w:Template:Citation Style documentation/usage vertical
w:Template:Citation Style documentation/usage vertical common
w:Template:Citation Style documentation/usage vertical end
w:Template:Citation Style documentation/usage vertical mid
w:Template:Citation Style documentation/web
w:Template:Citation Style documentation/whats new
w:Template:Cite web/doc
w:Template:Cl
w:Template:Code
w:Template:Crossref
w:Template:Crossreference
w:Template:Cs1
w:Template:Csdoc
w:Template:Date
w:Template:Diff
w:Template:Documentation
w:Template:Documentation subpage
w:Template:Em
w:Template:End
w:Template:Hatnote inline
w:Template:Hatnote inline/invoke
w:Template:High-use
w:Template:Infobox
w:Template:Lang
w:Template:Lua
w:Template:Main other
w:Template:Markup2
w:Template:Mbox
w:Template:MediaWiki URL rules
w:Template:NCJ
w:Template:Navbox
w:Template:NotRef
w:Template:Nowrap
w:Template:Ombox
w:Template:Para
w:Template:Paren
w:Template:Pipe
w:Template:Purge
w:Template:R
w:Template:R/ref
w:Template:Reflist
w:Template:Sandbox other
w:Template:Section link
w:Template:Sfnote
w:Template:Slink
w:Template:Str left
w:Template:Sup
w:Template:Tag
w:Template:TemplateData header
w:Template:Template link with link off
w:Template:Template other
w:Template:Template parameter usage
w:Template:Tl
w:Template:Tlf
w:Template:Tlg
w:Template:Tlx
w:Template:Trim
w:Template:UF-COinS
w:Template:Var
w:Template:Warning
w:Template:Wikipedia referencing
w:Template:Xt
w:Template:Xtd
w:Template:Yesno
w:Template:Yesno-no
w:Help:CS1 errors
w:Module:Anchor
w:Module:Arguments
w:Module:Check for unknown parameters
w:Module:Citation/CS1
w:Module:Citation/CS1/COinS
w:Module:Citation/CS1/Configuration
w:Module:Citation/CS1/Date validation
w:Module:Citation/CS1/Identifiers
w:Module:Citation/CS1/Utilities
w:Module:Citation/CS1/Whitelist
w:Module:Citation/CS1/styles.css
w:Module:Color contrast
w:Module:Color contrast/colors
w:Module:Cs1 documentation support
w:Module:Documentation
w:Module:Documentation/config
w:Module:Effective protection expiry
w:Module:Effective protection level
w:Module:Exponential search
w:Module:File link
w:Module:Hatnote
w:Module:Hatnote inline
w:Module:High-use
w:Module:Infobox
w:Module:Lang
w:Module:Lang/ISO 639 synonyms
w:Module:Lang/data
w:Module:Language/data/iana languages
w:Module:Language/data/iana regions
w:Module:Language/data/iana scripts
w:Module:Language/data/iana suppressed scripts
w:Module:Language/data/iana variants
w:Module:List
w:Module:Lua banner
w:Module:Message box
w:Module:Message box/configuration
w:Module:Navbar
w:Module:Navbox
w:Module:No globals
w:Module:Protection banner
w:Module:Protection banner/config
w:Module:Purge
w:Module:Section link
w:Module:String
w:Module:TableTools
w:Module:Template wrapper
w:Module:Tlg
w:Module:Transclusion count
w:Module:Transclusion count/data/C
w:Module:Unicode data
w:Module:Unicode data/scripts
w:Module:Yesno
w:Template:Citation Style documentation/edition
w:Template:Citation Style documentation/title
w:Template:Citation Style documentation/volume
w:Template:Cite news/doc
w:Template:High-risk
w:Template:Cite journal/doc
w:Template:Vcite
w:Template:'
w:Template:Citation Style documentation/chapter
w:Template:Citation Style documentation/chapterurl
w:Template:Cite book/TemplateData
w:Template:Cite book/doc
w:Template:Collapse bottom
w:Template:Collapse top
w:Template:Hover title
w:Template:Citation Style documentation/time
w:Template:Cite AV media/doc
w:Template:Pre2
w:Template:Abbr
w:Template:Cascade-protected template
w:Template:Category handler
w:Template:Citation/doc
w:Template:Citation Style documentation/sep comma
w:Template:Define
w:Template:Delink
w:Template:Edit
w:Template:Fake citation needed
w:Template:Fake fix
w:Template:Fix
w:Template:Fmbox
w:Template:For2
w:Template:Harv
w:Template:Harvard citation
w:Template:Harvard citation no brackets
w:Template:Harvid
w:Template:Harvnb
w:Template:Hatnote
w:Template:Nbsp
w:Template:Notice
w:Template:Protected page text
w:Template:Resize
w:Template:SfnRef
w:Template:Smaller
w:Template:Spaces
w:Template:Wikipedia help pages
w:Module:Category handler
w:Module:Category handler/blacklist
w:Module:Category handler/config
w:Module:Category handler/data
w:Module:Category handler/shared
w:Module:Delink
w:Module:Footnotes
w:Module:Footnotes/anchor id list
w:Module:Footnotes/anchor id list/data
w:Module:Footnotes/whitelist
w:Module:Namespace detect/config
w:Module:Namespace detect/data

Event Timeline

Tgr added a subscriber: Tgr.

You shouldn't insert content IDs manually, the import script will do that. If you do insert by hand, you'll probably end up with a stale NameTableStore cache.

You shouldn't insert content IDs manually, the import script will do that.

This was seemingly not happening with the imports

I'm presuming the list was from enwiki... So

So an untouched vagrant mediawiki master install.. Enable scribunto (otherwise it errors, lol), templatestyles and templatedata roles

Import the XML file

(03:24) vagrant@localhost:[wiki]> select * from content_models;
+----------+---------------+
| model_id | model_name    |
+----------+---------------+
|        2 | Scribunto     |
|        3 | sanitized-css |
|        1 | wikitext      |
+----------+---------------+
3 rows in set (0.00 sec)

So it all seems to have worked fine. Use {{cite web}} works without the reported error

I guess I need to try it on 1.35...

I guess I need to try it on 1.35...

Which I can't do on MediaWiki-Vagrant due to having PHP 7.2, but needing 7.3... Due to T256822: [EPIC] Upgrade MediaWiki-Vagrant to Debian Buster needing to happen first, I guess

Yay. I'll try on my dev vm when I've had some sleep... Having an LTS wiki around is probably helpful

I don't have vagrant, but I do have good notes on how to install MediaWiki with git clone, so I'll put the old (upgraded and possibly not working as expected due to a past error, usually on my part) aside and install clean 1.35 and get back to this thread if it turns out the MW was not in a consistent state or if the problem with the template imports still perists. Right now I have some work, but I will try to do this soon after I've done the work the boss needs.

You don't have to use vagrant (and neither do I), I just thought it would've been an easier way to spin up a wiki I can easily keep destroying ;)

MariaDB [wikidb_rel135]> select * from mw_content_models;
+----------+------------+
| model_id | model_name |
+----------+------------+
|        1 | wikitext   |
+----------+------------+
1 row in set (0.001 sec)

So that is a clean MW REL1_35 install. Clone TemplateStyles. comoser update --no-dev inside TemplateStyles. Enable Cite/Scribunto/TemplateData/TemplateStyles.

Add

require "$IP/includes/DevelopmentSettings.php";
$wgLogExceptionBacktrace = true;
$wgDebugDumpSql = true;
$wgDebugComments = true;

Import the XML file...

Screenshot 2020-11-30 at 02.42.43.png (634×1 px, 119 KB)

Importing pages...

    Template:Cite web 1 revision
    Template:Cite news 1 revision
    Template:Cite journal 1 revision
    Template:Cite book 1 revision
    Template:Cite AV media 1 revision
    Template:Citation 1 revision
    Template:!xt 1 revision
    Template:= 1 revision
    Template:AWB standard installation 1 revision
    Template:Anchor 1 revision
    Template:Category link 1 revision
    Template:Citation Style 1 1 revision
    Template:Citation Style documentation 1 revision
    Template:Citation Style documentation/agency 1 revision
    Template:Citation Style documentation/author 1 revision
    Template:Citation Style documentation/auto-date-format 1 revision
    Template:Citation Style documentation/coins 1 revision
    Template:Citation Style documentation/cs1 1 revision
    Template:Citation Style documentation/date 1 revision
    Template:Citation Style documentation/deprecated 1 revision
    Template:Citation Style documentation/display 1 revision
    Template:Citation Style documentation/editor 1 revision
    Template:Citation Style documentation/id1 1 revision
    Template:Citation Style documentation/id2 1 revision
    Template:Citation Style documentation/journal 1 revision
    Template:Citation Style documentation/language 1 revision
    Template:Citation Style documentation/lay 1 revision
    Template:Citation Style documentation/lead 1 revision
    Template:Citation Style documentation/lua 1 revision
    Template:Citation Style documentation/pages 1 revision
    Template:Citation Style documentation/publisher 1 revision
    Template:Citation Style documentation/quote 1 revision
    Template:Citation Style documentation/ref 1 revision
    Template:Citation Style documentation/registration 1 revision
    Template:Citation Style documentation/required 1 revision
    Template:Citation Style documentation/sep period 1 revision
    Template:Citation Style documentation/series 1 revision
    Template:Citation Style documentation/syntax 1 revision
    Template:Citation Style documentation/type 1 revision
    Template:Citation Style documentation/url 1 revision
    Template:Citation Style documentation/usage 1 revision
    Template:Citation Style documentation/usage common 1 revision
    Template:Citation Style documentation/usage full 1 revision
    Template:Citation Style documentation/usage vertical 1 revision
    Template:Citation Style documentation/usage vertical common 1 revision
    Template:Citation Style documentation/usage vertical end 1 revision
    Template:Citation Style documentation/usage vertical mid 1 revision
    Template:Citation Style documentation/web 1 revision
    Template:Citation Style documentation/whats new 1 revision
    Template:Cite web/doc 1 revision
    Template:Cl 1 revision
    Template:Code 1 revision
    Template:Crossref 1 revision
    Template:Crossreference 1 revision
    Template:Cs1 1 revision
    Template:Csdoc 1 revision
    Template:Date 1 revision
    Template:Diff 1 revision
    Template:Documentation 1 revision
    Template:Documentation subpage 1 revision
    Template:Em 1 revision
    Template:End 1 revision
    Template:Hatnote inline 1 revision
    Template:Hatnote inline/invoke 1 revision
    Template:High-use 1 revision
    Template:Infobox 1 revision
    Template:Lang 1 revision
    Template:Lua 1 revision
    Template:Main other 1 revision
    Template:Markup2 1 revision
    Template:Mbox 1 revision
    Template:MediaWiki URL rules 1 revision
    Template:NCJ 1 revision
    Template:Navbox 1 revision
    Template:NotRef 1 revision
    Template:Nowrap 1 revision
    Template:Ombox 1 revision
    Template:Para 1 revision
    Template:Paren 1 revision
    Template:Pipe 1 revision
    Template:Purge 1 revision
    Template:R 1 revision
    Template:R/ref 1 revision
    Template:Reflist 1 revision
    Template:Sandbox other 1 revision
    Template:Section link 1 revision
    Template:Sfnote 1 revision
    Template:Slink 1 revision
    Template:Str left 1 revision
    Template:Sup 1 revision
    Template:Tag 1 revision
    Template:TemplateData header 1 revision
    Template:Template link with link off 1 revision
    Template:Template other 1 revision
    Template:Template parameter usage 1 revision
    Template:Tl 1 revision
    Template:Tlf 1 revision
    Template:Tlg 1 revision
    Template:Tlx 1 revision
    Template:Trim 1 revision
    Template:UF-COinS 1 revision
    Template:Var 1 revision
    Template:Warning 1 revision
    Template:Wikipedia referencing 1 revision
    Template:Xt 1 revision
    Template:Xtd 1 revision
    Template:Yesno 1 revision
    Template:Yesno-no 1 revision
    Help:CS1 errors 1 revision
    Module:Anchor 1 revision
    Module:Arguments 1 revision
    Module:Check for unknown parameters 1 revision
    Module:Citation/CS1 1 revision
    Module:Citation/CS1/COinS 1 revision
    Module:Citation/CS1/Configuration 1 revision
    Module:Citation/CS1/Date validation 1 revision
    Module:Citation/CS1/Identifiers 1 revision
    Module:Citation/CS1/Utilities 1 revision
    Module:Citation/CS1/Whitelist 1 revision
    Module:Citation/CS1/styles.css 1 revision
    Module:Color contrast 1 revision
    Module:Color contrast/colors 1 revision
    Module:Cs1 documentation support 1 revision
    Module:Documentation 1 revision
    Module:Documentation/config 1 revision
    Module:Documentation/styles.css 1 revision
    Module:Effective protection expiry 1 revision
    Module:Effective protection level 1 revision
    Module:Exponential search 1 revision
    Module:File link 1 revision
    Module:Hatnote 1 revision
    Module:Hatnote inline 1 revision
    Module:High-use 1 revision
    Module:Infobox 1 revision
    Module:Lang 1 revision
    Module:Lang/ISO 639 synonyms 1 revision
    Module:Lang/data 1 revision
    Module:Language/data/iana languages 1 revision
    Module:Language/data/iana regions 1 revision
    Module:Language/data/iana scripts 1 revision
    Module:Language/data/iana suppressed scripts 1 revision
    Module:Language/data/iana variants 1 revision
    Module:List 1 revision
    Module:Lua banner 1 revision
    Module:Message box 1 revision
    Module:Message box/configuration 1 revision
    Module:Navbar 1 revision
    Module:Navbox 1 revision
    Module:No globals 1 revision
    Module:Protection banner 1 revision
    Module:Protection banner/config 1 revision
    Module:Purge 1 revision
    Module:Section link 1 revision
    Module:String 1 revision
    Module:TableTools 1 revision
    Module:Template wrapper 1 revision
    Module:Tlg 1 revision
    Module:Transclusion count 1 revision
    Module:Transclusion count/data/C 1 revision
    Module:Unicode data 1 revision
    Module:Unicode data/scripts 1 revision
    Module:Yesno 1 revision
    Template:Citation Style documentation/edition 1 revision
    Template:Citation Style documentation/title 1 revision
    Template:Citation Style documentation/volume 1 revision
    Template:Cite news/doc 1 revision
    Template:High-risk 1 revision
    Template:Cite journal/doc 1 revision
    Template:Vcite 1 revision
    Template:' 1 revision
    Template:Citation Style documentation/chapter 1 revision
    Template:Citation Style documentation/chapterurl 1 revision
    Template:Cite book/TemplateData 1 revision
    Template:Cite book/doc 1 revision
    Template:Collapse bottom 1 revision
    Template:Collapse top 1 revision
    Template:Hover title 1 revision
    Template:Citation Style documentation/time 1 revision
    Template:Cite AV media/doc 1 revision
    Template:Pre2 1 revision
    Template:Abbr 1 revision
    Template:Cascade-protected template 1 revision
    Template:Category handler 1 revision
    Template:Citation/doc 1 revision
    Template:Citation Style documentation/sep comma 1 revision
    Template:Define 1 revision
    Template:Delink 1 revision
    Template:Edit 1 revision
    Template:Fake citation needed 1 revision
    Template:Fake fix 1 revision
    Template:Fix 1 revision
    Template:Fmbox 1 revision
    Template:For2 1 revision
    Template:Harv 1 revision
    Template:Harvard citation 1 revision
    Template:Harvard citation no brackets 1 revision
    Template:Harvid 1 revision
    Template:Harvnb 1 revision
    Template:Hatnote 1 revision
    Template:Nbsp 1 revision
    Template:Notice 1 revision
    Template:Protected page text 1 revision
    Template:Resize 1 revision
    Template:SfnRef 1 revision
    Template:Smaller 1 revision
    Template:Spaces 1 revision
    Template:Wikipedia help pages 1 revision
    Module:Category handler 1 revision
    Module:Category handler/blacklist 1 revision
    Module:Category handler/config 1 revision
    Module:Category handler/data 1 revision
    Module:Category handler/shared 1 revision
    Module:Delink 1 revision
    Module:Footnotes 1 revision
    Module:Footnotes/anchor id list 1 revision
    Module:Footnotes/anchor id list/data 1 revision
    Module:Footnotes/whitelist 1 revision
    Module:Namespace detect/config 1 revision
    Module:Namespace detect/data 1 revision

Import finished!
MariaDB [wikidb_rel135]> select * from mw_content_models;
+----------+---------------+
| model_id | model_name    |
+----------+---------------+
|        2 | Scribunto     |
|        3 | sanitized-css |
|        1 | wikitext      |
+----------+---------------+
3 rows in set (0.000 sec)

I definitely can't replicate this as is...

NameTableStore is pretty self-contained, callers (such as the import logic) should have no idea whether the record exists already or not. They just exchange the model name for an ID which might or might not be freshly created. So I would suspect a database-level issue here (writes fail, but somehow without MediaWiki noticing it, or maybe there is some over-eager exception handling happening; the rest of the import works due to NameTableStore having cached the model name / ID pair, but as soon as that cache expires things blow up).

Hi and thanks to you helpful folks.

I just yesterday reinstalled my testwiki with git, and in the fresh installation of 1.35 the bug no longer reproduces.

Must have been a not-so-well-handled upgrade down the road on my testwiki.

After having successfully tested on a testwiki I have been able to enable the citation templates on the wikis where they are needed.

Sorry to have bothered you. Issue wasn't a MediaWiki issue in the end.