Watching the import aftermath on the job queue over the past week or so via the showJobs.php maintenance script, an import of an account with very few (less than five) edits/actions in the database generates over 13,000 reattributeImportedEdits jobs, which seems completely disproportionate, and is probably the cause of observed/reported difficulties with importing accounts for Yugipedia.
Description
Description
Details
Details
| Subject | Repo | Branch | Lines +/- | |
|---|---|---|---|---|
| Make edit attribution actor-aware | mediawiki/extensions/MediaWikiAuth | master | +384 -120 |
Related Objects
Related Objects
Event Timeline
Comment Actions
What version of MediaWiki are you importing into (target wiki where MWA is installed)?
If 1.31 or later, what is the value of $wgActorTableSchemaMigrationStage on your wiki?
Comment Actions
It looks like the current behavior of the extension is to generate jobs for each block of 300 edits in the wiki, and then try to reattribute later within each block. So, if your wiki has 300,000 edits, each import would generate 1,000 jobs. This means that the majority of the jobs will actually be doing absolutely nothing, and it just serves to bloat the job queue.
There is certainly a better way of handling this, although I can't give any ETA of when I'll be able to get around to fixing this issue (as well as adding support for the actor migration). I have a couple other projects in front of this one.
Comment Actions
Aah, so with 4.125 million edits, it results in 13,750 jobs being generated. This definitely lines up with what I've been seeing.
Comment Actions
Change 559286 merged by jenkins-bot:
[mediawiki/extensions/MediaWikiAuth@master] Make edit attribution actor-aware
Comment Actions
We now generate jobs proportional to the number of edits that need to be reattributed instead of total number of edits, which should reduce database impact by a lot. Some extra work is needed at import time to do this, however.