This has been split off as it's own ticket as it requires more investigation: https://phabricator.wikimedia.org/T413096

Available here: https://www.kaggle.com/datasets/wikimedia-foundation/wikipedia-structured-contents!

@JArguello-WMF updated the description to reflect the dark launch date

side note: summary endpoint also includes some special characters and styling: https://en.wikipedia.org/api/rest_v1/page/summary/List_of_The_Elusive_Samurai_chapters

Link to tool in description doesn't work for me anymore, but this one does: https://internetarchive.github.io/iare/?url=https://en.wikipedia.org/wiki/Lionel_Messi&method=IABOT

and https://github.com/internetarchive/wiki-references-db, https://github.com/internetarchive/wiki-references-extractor

code: https://github.com/internetarchive/iari

More info on API: https://xtools.wmcloud.org/api and https://www.mediawiki.org/wiki/XTools/API

Ok, thanks a lot for the quick help and clarification, @cscott !

great, thanks!

More feedback:

Thanks @cscott!

Thanks @ABreault-WMF and @cscott! The expected behaviour is spaces?

Thanks @cscott, is the different order due to the same issue as the missing reference? For this particular page, the missing reference was reference nr 9, but the order is already different from nr 3, and the inline citations are now pointing to different references in legacy html vs parsoid for the same statement (see screenshot).

and
sections.has_parts.has_parts.has_parts.has_parts.name
sections.has_parts.has_parts.has_parts.has_parts.has_parts.links.images
sections.has_parts.has_parts.has_parts.has_parts.has_parts.has_parts
sections.has_parts.has_parts.has_parts.has_parts.has_parts.has_parts.links

Albert also added missing field infoboxes.has_parts.images

More feedback:

First changes are done.
Feedback from Albert:
After implementing the schema, I am finding some additional issues because the data is not aligned with the schema. For example:

• I had to fix the names false and true to 'false' and 'true' (with quotes)
• I had to rename the field no_index to noindex (without underscore)
• I had to add the missing field event.date_published

Also I discovered the root field in_language is duplicated.
And now I am also facing a new missing field called "images": I am investigating in which super-field.
So I was wondering if you have a newer version of the complete schema. Otherwise, I should continue trying locally until I get the complete schema.