Page MenuHomePhabricator

change enwiki database dump intervals (15th day of month instead of 20th)
Closed, DeclinedPublic

Description

A user wrote at https://meta.wikimedia.org/wiki/Tech#enwiki_database_dump_intervals:

I run a bot to update geodata tags on enwiki that works by parsing enwiki database dumps. At the moment, we have two dumps of enwiki per month; one is set off on the 1st of the month, and the other is set off on the 20th. This means that we have a dump interval of alternating periods of (roughly) 10 and 20 days, meaning that one bot run does roughly twice the work of the other, and the maximum lag between data change and bot action is about 20 days.

Would it be possible to set the second dump off on the 15th (or perhaps 16th) of each month instead of the 20th, so that the dump intervals are a bit more uniform? The Anome (talk) 21:17, 15 August 2022 (UTC)

Is it possible? What's the story behind picking the 20th of the month (I am sure there is one)?

Event Timeline

The first sql/xml dumps run (1st of every month) is a full run, this means it contains all historical data and takes longer to dump. Currently we still have the wikidata dumps ongoing, and expect it to complete tomorrow. This gives us a margin of a few days in case there are errors causing us to delay or rerun some parts.

The second run starts on the 20th and is a partial run (current revision only) which is why it finishes well before the end of the month, giving us again a few days in case we need to troubleshoot or rerun some parts.

While the specific wiki, enwiki, does complete well before the 15th, splitting up the wikis so that some are dumped on a 1st and 15th schedule and some are dumped on a 1st and 20th schedule would add more complexity than we want to manage.

I hope this helps to answer your questions, and we would be happy to respond if you need further information.

Thank you

@Hokwelum Thank you very much for the prompt and detailed answer. I forwarded it to the original user on the wiki talk page.

@Hokwelum Thank you very much for the prompt and detailed answer. I forwarded it to the original user on the wiki talk page.

Is it ok to go ahead and close this task now, @Dzahn ?

Dzahn claimed this task.

@ArielGlenn They said "Thank you. That all makes sense. ". Yes, we can close it. I pointed out they can also leave their own comments here (wiki user should work on phab). Thank you.

Dzahn changed the task status from Resolved to Declined.Aug 19 2022, 12:37 AM