Page MenuHomePhabricator

Mass bigdeletion scheduled for sr.wikinews
Open, NormalPublic

Description

See:

Most of them pages have thousands of revisions each and are a bunch. With the batched deletion feature now live I guess it might not be a problem but I'd like to ping you nonetheless because I don't want to cause db issues or high replag while doing this.

I'm thinking about using a pywiki script. Perhaps a waiting time of 30 seconds between deletions with a maxlag parameter of 3 would be db friendly?

Please let me know. Thank you.

Event Timeline

Restricted Application added subscribers: Petar.petkovic, Aklapper. · View Herald TranscriptDec 19 2018, 9:31 PM

Do you happen to know how big a batch is?

Do you happen to know how big a batch is?

I'm not sure I understand the question. The batch of pages to delete is actually listed above and consist of 350 pages, most of them with more than 5,000 revisions each. Thanks.

Sorry for not being clear enough - I mean from a pure DB point of view, the internal of those batches, as in: how many rows per DELETE would be issued on the database :-)
Can we manually try to delete one of the pages with the highest number of revisions first? instead to do all of the pages at once so we can see the impact?
Ideally we should not do it between 24th and 1st as there will be a lot less people around due to holidays

Marostegui triaged this task as Normal priority.Dec 20 2018, 6:40 AM
Marostegui moved this task from Triage to Blocked external/Not db team on the DBA board.

Right, my question is answered at T198176#4724935
I think with that and your additional suggestion of waiting 30 seconds + maxlag of 3 we should be fine. However, it doesn't hurt to do a first manual run to double check? Would you be ok with that?

@Marostegui Unfortunately I'll have to do all this by hand given that the pywiki script fails due to returning an error message which is not an error (T212356). I can find the page with most revisions and coordinate with you to monitor the screens if you're okay with that as well. Thanks.

@Marostegui Unfortunately I'll have to do all this by hand given that the pywiki script fails due to returning an error message which is not an error (T212356). I can find the page with most revisions and coordinate with you to monitor the screens if you're okay with that as well. Thanks.

Sounds good to me - ping me once you've got it so we can arrange a day/time for it

@Marostegui Given that db67de7fad isn't yet live in WMF branches, perhaps we could wait until it is there so we can do it with the bot, notwithstanding the supervision of the ops/dba team to avoid lag, db locks, etc. What do you think?

Sounds good to me!

@Marostegui Given that db67de7fad isn't yet live in WMF branches, perhaps we could wait until it is there so we can do it with the bot, notwithstanding the supervision of the ops/dba team to avoid lag, db locks, etc. What do you think?

I think to we can wait until deployment of db67de7fad.. This is not urgent.

@Marostegui Given that db67de7fad isn't yet live in WMF branches, perhaps we could wait until it is there so we can do it with the bot, notwithstanding the supervision of the ops/dba team to avoid lag, db locks, etc. What do you think?

I think to we can wait until deployment of db67de7fad.. This is not urgent.

I think to this is deployed. I will try to delete pages again which I can. I will update list (https://sr.wikinews.org/wiki/User:MarcoAurelio/ToDel) after it.

I can give this another try.

I can give this another try.

I will tell you when you can try. I will start now.

Okay.

@MarcoAurelio I tryed, but I still have problems. As generator for pages which should be deleted use contributions from user Millbot-Weather. As summary use [[:m:Special:Permalink/18727625#srwikinews-delreq]] ([[:phab:T212346|T212346]]). Thanks!

I see that you're being able to delete some of those pages. Please note that pages with more than one thousand of revisions are not deleted instantly but via a JobQueue job so it can take some time until the deletion happens. I'll check it out later which pages you were not able to delete and try on them.

I see that you're being able to delete some of those pages. Please note that pages with more than one thousand of revisions are not deleted instantly but via a JobQueue job so it can take some time until the deletion happens. I'll check it out later which pages you were not able to delete and try on them.

I deleting categories related to things for which we doing this now. These categories are made by second bot. I will generate new list on your page. I will let you know later when I delete all pages which I can.

Please note that pages with more than one thousand of revisions are not deleted instantly but via a JobQueue job so it can take some time until the deletion happens.

I saw it.

Zoranzoki21 added a comment.EditedJan 11 2019, 1:32 PM

@MarcoAurelio I generated you new list for first part of deletion. See https://sr.wikinews.org/wiki/Корисник:MarcoAurelio/ToDel

You can start when you feel conform to do it. When you delete all these pages, I will generate new list again. I did not want to exaggerate with a list because of possible crash in browser for loading.

Thank you. I'll try to delete those pages then. To avoid flooding recent changes, do you think I can get a local bot flag at sr.wikinews? Ping @dungodung as one of the local bureaucrats.

Thank you. I'll try to delete those pages then. To avoid flooding recent changes, do you think I can get a local bot flag at sr.wikinews? Ping @dungodung as one of the local bureaucrats.

You will get it. I will ask another bureaucrat to give you it.

I don't see a need for deleting those pages.

I don't see a need for deleting those pages.

Thanks for your comment @dungodung - On meta they said there was a community discussion. Nevertheless, until this is clarified, I'll abstain from deleting anything.

As you can see on [[sr:n:Викивести:Форум]] there was no discussion about this at all. Also, I wouldn't consider a few disparate users that contribute here and there a community...

As you can see on [[sr:n:Викивести:Форум]] there was no discussion about this at all. Also, I wouldn't consider a few disparate users that contribute here and there a community...

There is not active community. You no know what wiki project should be. For you only is importmant number of articles. Ende. Too you no know to say something good for users. When you make mistakes, noone must not tell you something bad to you. When we make mistakes, you are first to shout. @Ranko95, @Novak_Watchmen, @Acamicamacaraca, @MareBG and I were FOR. Perhaps the Wikimedia of Serbia will also introduce "seminar works" there. 😄

Please don't troll here.

Please don't troll here.

Again:

When you make mistakes, noone must not tell you something bad to you. When we make mistakes, you are first to shout.

Zoranzoki21 changed the task status from Open to Stalled.Jan 11 2019, 5:24 PM

I will start public discussion so..

@Zoranzoki21 and @dungodung, as well as other subscribers- Phabricator is not the place for this kind of discussion- that should go to wiki. As @MarcoAurelio said, no deletion will happen until that happens, and even if some deletion happened already, it can be recovered. Please solve disputes on wiki, and only return to this ticket when a consensus is reached with a decision we can apply.

@Zoranzoki21 and @dungodung, as well as other subscribers- Phabricator is not the place for this kind of discussion- that should go to wiki. As @MarcoAurelio said, no deletion will happen until that happens, and even if some deletion happened already, it can be recovered. Please solve disputes on wiki, and only return to this ticket when a consensus is reached with a decision we can apply.

Okay, I will start public discussion. Until end of discussion, nothing will not be done.

@dungodung, @Zoranzoki21: This tone is not welcome here. Please read https://www.mediawiki.org/wiki/Bug_management/Phabricator_etiquette if you would like to be active on Wikimedia Phabricator. Thank you for keeping this a respectful place.

@dungodung, @Zoranzoki21: This tone is not welcome here. Please read https://www.mediawiki.org/wiki/Bug_management/Phabricator_etiquette if you would like to be active on Wikimedia Phabricator. Thank you for keeping this a respectful place.

@Aklapper Being blocked because of personal attitude... I understand the rules, but I answered the question only.

I started "public discussion": https://sr.wikinews.org/wiki/Викивести:Форум#Дискусија_о_брисању_садржаја_везаног_за_временску_прогнозу

Until now, noone was not opposed 😃. In 7 days we will have proof that I did not do anything on my own.

Zoranzoki21 changed the task status from Stalled to Open.Jan 17 2019, 2:14 PM

Noone was not opposed, reopening.

What's the plan for this? Is this T212346#4845733 ready?

What's the plan for this? Is this T212346#4845733 ready?

It is live on wmf wikis, this should be done, we have consensus.

Do you want to give it a go on Monday?

Do you want to give it a go on Monday?

It is ok for me.

Cool, comment on this task before you are ready to start :)

Cool, comment on this task before you are ready to start :)

Ok, until Monday I will try to delete all pages which have less than 5000 revisions. Before Monday I will generate full list of pages which should be deleted.

If possible comment here before starting so we can confirm it is ok to proceed.
Just in case there is something going on at that moment (ie: incident, an unexpected maintenance etc)

Thanks!

If possible comment here before starting so we can confirm it is ok to proceed.
Just in case there is something going on at that moment (ie: incident, an unexpected maintenance etc)

Thanks!

Sure.

I started with working, replag is possible.

I thought we agreed on doing it on Monday?

I thought we agreed on doing it on Monday?

@Marostegui

Ok, until Monday I will try to delete all pages which have less than 5000 revisions. Before Monday I will generate full list of pages which should be deleted.

If I shouldn't work this now, I will stop.

Sorry, I didn't get the less than 5000 revisions part :-)
I think we should probably wait till Monday for the whole thing as there are not many people around on a Saturday morning :-)
Just being safe here!
Thank you!

Sorry, I didn't get the less than 5000 revisions part :-)
I think we should probably wait till Monday for the whole thing as there are not many people around on a Saturday morning :-)
Just being safe here!
Thank you!

With this request I requested deleting pages which have more than 5000 revisions because administrators can't delete these pages but stewards can.. @Marostegui Can you suggest me time for which you feel conform?

From 8am UTC on Monday would work for me!

From 8am UTC on Monday would work for me!

For me 2pm UTC+1 because I'm at school in the morning..

Good for me too :-)

Good for me too :-)

Ok :) Thanks.. @MarcoAurelio For you?

I think I'll be avalaible on Monday, but I will coordinate with @Marostegui directly when steward intervention is required.

I think I'll be avalaible on Monday, but I will coordinate with @Marostegui directly when steward intervention is required.

@MarcoAurelio I deleted all pages which I can. You have 240 pages now..Your account have bot flag (https://sr.wikinews.org/wiki/Посебно:КорисничкаПрава/MarcoAurelio) so you will not spam recent changes.

I will track this progress in Monday.

I am going to defer the deletion of those pages until the maintenance script being used to do the actor migration finishes everywhere, to avoid potentially adding more lag to the databases. Discussed and agreed with DBA.

I am going to defer the deletion of those pages until the maintenance script being used to do the actor migration finishes everywhere, to avoid potentially adding more lag to the databases. Discussed and agreed with DBA.

Ok. No problems for me.

I am going to defer the deletion of those pages until the maintenance script being used to do the actor migration finishes everywhere, to avoid potentially adding more lag to the databases. Discussed and agreed with DBA.

As I see, we can resume with working on this. Do you agree?

Okay. I don't feel too well yet (flu, etc.) but I'll have a look today.

MarcoAurelio added a comment.EditedFeb 4 2019, 8:54 PM

This is annoying. Now Pywikibot complains each time ERROR: Login failed (Aborted). I suspect this is because delete.py tries to log me in every time before making a deletion (T212346).

Can we have deleteBatch.php handle this for us?

Could I have my botflag at sr.wikinews removed, please? Until T122705 is resolved I won't be able to handle this, and at this point I think that the maintenance script path will be a better approach. Thank you.

Could I have my botflag at sr.wikinews removed, please? Until T122705 is resolved I won't be able to handle this, and at this point I think that the maintenance script path will be a better approach. Thank you.

It will be removed.

Could I have my botflag at sr.wikinews removed, please? Until T122705 is resolved I won't be able to handle this, and at this point I think that the maintenance script path will be a better approach. Thank you.

It will be removed.

It is removed before 3 weeks. Can we have deleteBatch.php run for this?

removing the DBA tag from here - I will remain subscribed so we can coordinate once this is moved forward