Create a structured list of when a Wikimedia project is created, and if applicable, the closure date as well.
Sources for project creation
Sources for project closure
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Declined | nshahquinn-wmf | T369207 Canonical Data | Update, Maintain, Improve | |||
| Open | None | T336999 Create a structured list of Wikimedia projects' creation and closure dates |
Learnings from the progress made during the hackathon:
Wikimedia project creation date
The git files were created on 2012-02-24, which is the reason for many wikis having their date recorded as 2012-02-24. This source can only be used for wikis created after 2012-02-24. For wikis created before, three sources are under consideration, based on the timeline.
@KCVelaga_WMF have you had a chance to work on the last pieces of this? Totally fine if not, but if you want to hand it over, I'm happy to take it on 😊
@nshahquinn-wmf thanks for checking on this.
I have moved the repository under wikimedia-research on GitHub: https://github.com/wikimedia-research/wikimedia_project_creation_closure_dts. I was able to complete the creation dates part, but not the closure dates. I don't think I will be able to get to it before the end of Q1, so if you can take over, that's wonderful. I am happy to collaborate, for thinking through the approaches, reviews and consolidating everything. I will go ahead and assign the task to you.
Here are some updates:
I just worked on a Wikipedia 25-related request from the WMF Communications department for:
I was able to provide the first (with a bunch of caveats around the precision of the earliest data), but not the second and third.
In the process, I ended up digging into the data that @KCVelaga_WMF prepared. Here's a current brain dump from the experience.
For terminology, see mw:MediaWiki_history.
For wikis that started off using UseModWiki, I don't think there's any structured way to get the creation date (or first article). My understanding is that the conversion process from UseModWiki (to either Phase II or MediaWiki) did not preserve history. Although some of that lost history has been restored ad-hoc, generally, I think we have to assume that is it lost.
For these wikis, the lowest revision ID will correspond to the first revision on the newer software, with old content imported later, and the lowest revision timestamp will either correspond to the same revision or to a revision imported not during the conversion process, but later using the XML import feature. That imported revision might have been made on the same wiki's UseModWiki install, but as or more likely, it was imported from a different wiki.
For these wikis, likely the only way to get the answer is manual historical work (e.g. digging through mailing lists and old wiki pages for references to the wikis being created). This would probably be able to get us the answer in most cases, but there will likely still be (1) some cases where we can't confirm the exact date and only know the range of, say, a couple weeks and (2) some ambiguity about what constitutes "creation".
Ultimately, we could probably get to sensible answers, but it would take a significant amount of historical work and judgement calls. If we opened a community discussion/call for evidence, we would probably get good answers and a lot of participation, but it would be a significant chunk of work to manage it.
For wikis which didn't start off using UseModWiki, I believe the answer is simpler: the time of the revision with the lowest ID. This seems to produce as good results or better results than any other method. I took 7 wikis where there was big discrepancy between the "least revision" and KC's results and found the creation date historically, and in all 7 cases, the "least revision" was accurate (and the other method was not).
Unfortunately, there also doesn't seem to be an easy way to distinguish the two groups (wikis which started off with UseModWiki and wikis which didn't). When wikis were converted from UseModWiki, content was imported to the new software using the username "Conversion script". As far as I can tell, this username has not been used for any other system activity, so I thought its presence in a wiki's history was a sure sign that it started off with UseModWiki.
Alas, it turns out that it shows up in other wikis too because those conversion revisions have been imported elsewhere, and identifying imported revision is not at all easy (see T22148).
There's actually a wikiBirthday maintenance script that uses the "timestamp of minimum rev ID" method which I found to be the best option despite it being mostly wrong for UseModWiki-first wikis.
In the WMF Slack, @Michael just asked about getting wiki creation dates for some newer wikis to understand why they're missing a particular table (T414600).
This is a case where it would be helpful for us to provide the creation date for only Phase II– or MediaWiki–first wikis, even if the data has to remain null for UseModWiki-first wikis. The main blocker for that is figuring out how to identify UseModWiki-first wikis.