The new wikipedia dump format drops the pages if they are edited between the first of the month and the time the dump happens.
The old dumps work.
I tried to give enough information to make this reproducible.
I noticed movies like Supergirl (2026) and The Devil Wears Prada 2 were missing from the dump.

I can use the old dumps, but this is a pretty bad bug (as all the popular/editted pages) get dropped.

Tom

I hit this independently on jawiki (2026-07-01 export) before finding this task,
so I can confirm the issue is real and not enwiki-specific.

I extracted all ns=0 pages from the two published jawiki files (p1p2517953 and
p2517954p5262960; both well-formed, _SUCCESS present, file dates 06-Jul, i.e. a
~5 day cutoff-to-generation window) and checked every wikilink target that could
not be resolved within the dump against the Action API.

Result: 18,287 currently existing pages with page ids inside the covered ranges
are absent from the export (about 1.2% of jawiki articles). Only 9 unresolved
titles were created after the snapshot and 1 was deleted since.

Example: 日本 (Japan, pageid 4821051) is absent, while ids 4821050 and 4821052
are both present in p2517954p5262960, so this is not file truncation.

The missing set is heavily skewed toward frequently edited pages (日本,
アメリカ合衆国, most prefectures, major countries), which is what the
"edited between cutoff and generation" mechanism predicts, since pages edited
daily always fall in that window. simplewiki 2026-07-01 checked the same way
had no missing prominent pages.

I can attach the full title/pageid list (TSV) if useful.

I confirm this bug (I just detected it on the French wiktionary dump).

As stated this affects ALL namespaces and is a MAJOR problem.

For instance, the 2026-07-01 dump of French wiktionary lacks the Module:lexique page because it has been modified July 3rd. This module is used by almost ALL French wiktionary pages. With this bug, any use of the dumps will be impossible because of missing templates/Modules. The probability of an edit on such pages in a 5-6 days window is quite high.

Expected behaviour:

1. Do not use cutoff date (export page as it is at the time of the effective export). Drawback: the versions of the pages may be incoherent (especially Templates/Modules)
2. Do extract the cutoff version of pages even if edited in the mean time (i.e. do not take the latest version at the time of dump, but the latest version dated before the cutoff)

Hey all, just acknowledging here that we can reproduce this issue, and that indeed it affects all MediaWiki Content Current File Exports.

We are working on a fix.

xcollazo opened https://gitlab.wikimedia.org/repos/data-engineering/airflow-dags/-/merge_requests/2586

Draft: mw_content_xml_export: Fix current export dropping edited pages

Change #1332814 had a related patch set uploaded (by Xcollazo; author: Xcollazo):

[analytics/refinery@master] File Export: HQL for point-in-time snapshot of mediawiki content

https://gerrit.wikimedia.org/r/1332814

Change #1332814 merged by Xcollazo:

[analytics/refinery@master] File Export: HQL for point-in-time snapshot of mediawiki content

https://gerrit.wikimedia.org/r/1332814

Exports had just started since it is Sept 1. We are pausing mediawiki_content_current exports generation as we are testing a fix for this issue. Some exports did finish already so we will likely delete them and rerun.

xcollazo merged https://gitlab.wikimedia.org/repos/data-engineering/airflow-dags/-/merge_requests/2586

mw_content_xml_export: Fix current export dropping edited pages

Did the following in prod:

1. Paused both mediawiki_content_history and mediawiki_content_current.
2. Deployed https://gitlab.wikimedia.org/repos/data-engineering/airflow-dags/-/merge_requests/2586
3. Deleted old Airflow Variable main/dags/mediawiki/content/export/mw_content_xml_export_dags.py so that it would re-generate with new values.
4. Reviewed everything looks good in Airflow. Unpaused mediawiki_content_history to continue as is. Cleared run for mediawiki_content_current, to start all dumps again for Sept 1 run.

Will continue observing.

Some details about the issue:
The mediawiki_content_current export was based on an internal table wmf_content.mediawiki_content_current_v1. That table stores exactly one row per page: its most recent revision. The export then filtered it with revision_dt < cutoff.

For any page edited between the snapshot cutoff and the XML generation run, the only stored row is the post-cutoff revision. The filter matched nothing, and the page was omitted from the output entirely, rather than falling back to its latest pre-cutoff revision.

@Tom_costello_ireland's hypothesised mechanism in the task description was right. Thank you for the precise report. Thanks also to @Miyajastu for the independent jawiki measurement and to @Dodecaplex for the frwiktionary Module:lexique case — we used both as test cases for the fix.

The fix:
The export no longer reads mediawiki_content_current_v1. It now builds a point-in-time table from wmf_content.mediawiki_content_history_v1, which is an internal table that retains every revision, and for each page selects the most recent revision dated before the cutoff. That is the second of the two behaviours @Dodecaplex described: pages are exported at their as-of-cutoff revision, not skipped and not advanced to a post-cutoff revision.

Verification against all wikis at cutoff 2026-07-01:

• The corrected snapshot holds 695,457,341 pages. The old code path would have published 652,541,720.
• frwiktionary Module:lexique is present, at its pre-cutoff revision from 2025-12-28.

The first corrected export will be the 2026-09-01 run.

Corrected exports for the reported wikis are now available:

enwiki: https://dumps.wikimedia.org/other/mediawiki_content_current/enwiki/2026-09-01/xml/bzip2/
jawiki: https://dumps.wikimedia.org/other/mediawiki_content_current/jawiki/2026-09-01/xml/bzip2/
frwiktionary: https://dumps.wikimedia.org/other/mediawiki_content_current/frwiktionary/2026-09-01/xml/bzip2/

@Tom_costello_ireland, @Miyajastu and @Dodecaplex: if you have a chance to verify the respective export of interest, that would be great.

Hi @xcollazo

Thanks for the fix, I confirm that the number of elements I extract each months from the dump is now raising at its normal rate in all wiktionary dumps I use (27 languages).

This was drastically better for English and French that have a very high number of edits.

Thanks again,

Hi, @xcollazo

I can confirm that this has fixed the "articles edited after snapshot are missing from dump" issue, but it seems there's a regression of the "moved articles don't have any page text" issue previous reported/fixed in T417596.

wget https://dumps.wikimedia.org/other/mediawiki_content_current/enwiktionary/2026-09-01/xml/bzip2/enwiktionary-2026-09-01-p6p12034212.xml.bz2
bzcat enwiktionary-2026-09-01-p6p12034212.xml.bz2 | grep '^      <text bytes="[^"]*" sha1="[^"]*" xml:space="preserve"></text>' | wc -l
1244

Sample of an empty page misclassified as a redirect in the ContentFileExport

bzcat enwiktionary-2026-09-01-p6p12034212.xml.bz2 | grep -B1 -A20 '<title>Template:en-intj</title>'
  <page>
    <title>Template:en-intj</title>
    <ns>10</ns>
    <id>463225</id>
    <revision>
      <id>87661056</id>
      <parentid>53167838</parentid>
      <timestamp>2025-10-31T07:10:17Z</timestamp>
      <contributor>
        <username>Benwing</username>
        <id>172432</id>
      </contributor>
      <minor />
      <comment>Benwing moved page [[Template:en-interj]] to [[Template:en-intj]]: Standardize name: standardize on shorter, easier to type name</comment>
      <origin>53167838</origin>
      <model>wikitext</model>
      <format>text/x-wiki</format>
      <text bytes="91" sha1="o74nl7xq9tkj2pu0e44uyb9gnnklk74" xml:space="preserve"></text>
      <sha1>o74nl7xq9tkj2pu0e44uyb9gnnklk74</sha1>
    </revision>
  </page>

In T431872#12304587, @JeffDoozan wrote:

Hi, @xcollazo

I can confirm that this has fixed the "articles edited after snapshot are missing from dump" issue, but it seems there's a regression of the "moved articles don't have any page text" issue previous reported/fixed in T417596.

@JeffDoozan thanks for the report. I will close this particular ticket and reopen T417596 for investigating your finding.